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Demodulation of Wideband, Low-Power 
FM Signals* 


By SIDNEY DARLINGTON 
(Manuscript received October 3, 1963) 


Some theoretical aspects of the demodulation of wideband, low-power 
FM signals are discussed. It 1s assumed that a band-limited, continuous, 
analog signal is supplied to the modulator and is recovered to a fidelity 
suitable for television, telephone, or carrier telephone. Much of the paper 
assumes that the baseband signal ts sampled and clamped before it is applied 
to the frequency modulator. The combination has been called PAM-FM 
and 1s characterized by a piecewise constant transmitted frequency. 

PAM-FM can be demodulated by spectrum analysis means not suitable 
for continuously varying frequencies. It is shown that a spectrum generator 
can be derived from the techniques of radar pulse compression, and ts equiv- 
alent to an infinite set of correlators or matched filters plus means for scan- 
ning their terminals. 

The spectrum analysis circuit forms are compared with demodulators 
using frequency detectors, with and without FM feedback, in regard to the- 
oretical notse sensitivities. The theoretical sensitivities are quite similar for 
spectrum analysis and FMFB under conditions assumed. The comparisons 
disclose that frequency detectors (followed by filters) enjoy a disguised but 
efficient use of a differential phase coherence which 1s a characteristic of 
FM signals. A combination of spectrum analysis and frequency detection 
ts described which has some of the theoretical advantages of both. | 


I. INTRODUCTION 


This paper discusses some theoretical aspects of the demodulation 
of wideband, low-power frequency modulated signals. A wide trans- 


* Parts of the material of this article were discussed by the author in lectures 
at the University of California at Berkeley during May, 1963 
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mitted bandwidth permits a saving in power. Frequency modulation 
implies a constant power level, which makes peak power identical with 
average power. It is advantageous, for example, when the practical 
restrictions on peak power determine system power levels rather than 
restrictions on average power. 

More specifically, the paper is concerned with I'M systems subject 
to the following external requirements: A band-limited, continuous, 
analog signal is supplied to the input of a coder or modulator, which 
produces the transmitted signal. A demodulator reproduces the original 
baseband signal to a fidelity suitable for a television channel, a telephone 
channel, or a carrier system combining a number of telephone channels. 
For such purposes, for example, the average errors in the output must 
be more than 40 db below the baseband signal. It is assumed that a large 
I'M index is used, to conserve signal power. These conditions are im- 
plicit in many of the conclusions. They will be referred to collectively 
as “‘the conditions assumed here.” 

Several different techniques and circuit forms are compared. The 
comparisons are concerned primarily, but not exclusively, with sensi- 
tivities to noise. Conventional FM receivers and circuits using FM 
feedback (FMFB) are included. However, more attention is paid to 
techniques which are closer to (but significantly different from) so-called 
frequency shift keying (ISK), a well-known method of data transmis- 
sion. Thus banks of correlators or matched filters appear in some of 
the proposed circuits, somewhat (but not exactly) as in FSK systems. 
Alternatively, the correlators or matched filters can be replaced by 
circuits resembling the pulse compressors of so-called Chirp radars,” 
and one (but not the only) purpose of the paper is to note how it can 
be done. 

Circuits of different kinds are compared not only among themselves 
but also with theoretical bounds derived from general information 
theory. Thus the paper draws on four major disciplines within the gen- 
eral field of communication theory and practice, namely: conventional 
FM and FMFB, discrete data transmission, pulse compression radars, 
and information theory. 

An expert in any one of the four disciplines may find some of the 
discussion quite familiar, and perhaps superfluous. However, it is un- 
likely that many readers will be thoroughly familiar with the pertinent 
parts of all the disciplines. Hence a somewhat tutorial approach has 
been adopted. However, some of the relations between disciplines and 
some of the circuit forms appear to be novel. 

The purpose of the paper is to describe and compare the various 


FM DEMODULATION 341 


techniques and circuit forms in simple terms. Mathematical proofs are 
outside the intended scope. Except in the Appendix, only the simplest 
formulas are stated explicitly, and circuits are represented only by simple 
block diagrams. A complete analysis is long, tedious, and mathematically 
uninteresting; a good deal of it differs only in detail from established 
applications to other problems. Some of the circuit forms have not 
actually been built; the block diagrams can be filled in with circuit 
details in many different ways, and best ways have not all been deter- 
mined. The Appendix outlines very briefly some analytical and circuit 
details, which may be needed for an appreciation of some of the conclu- 
sions. 


II. DEMODULATION BY SPECTRUM ANALYSIS 


Much of the paper concerns systems in which the analog baseband 
signal is sampled, as part of the initial modulation, but is not quantized. 
Fig. 1 is a corresponding block diagram. Each sample is clamped during 
the sample interval, and is supplied to a frequency modulator. Then 
the transmitted frequency is constant over each sample interval, but 
changes from interval to interval. Curve B of Fig. 2 illustrates the 
variation in frequency with time. It differs from frequency shift keying 
in the following way: The transmitted frequency may be anywhere in a 
continuum of frequencies; it is not restricted to a finite number of dis- 
crete frequencies. The distinction has important repercussions through- 
out the paper. 

If the sample interval is no greater than the Nyquist interval of the 
baseband bandwidth, the sampling destroys no information (at least in 
principle). It is assumed here that the sample interval equals the Ny- 
quist interval. 

Referring again to Fig. 1, the sequence of clamped samples at the 
input of the frequency modulator may be called a pulse amplitude 
modulation, or PAM representation of the original signal (with no gaps 
between the pulses). The corresponding output of the frequency modu- 
lator has been called PAM-FM.} It is a known means of adapting time 
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Fig. 1— Block diagram of a PAM-FM modulator. 
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division multiplex to frequency modulation.* (For multiplexing, signal 
samples are clamped for only fractions of sample intervals and are 
interleaved with the samples from other channels ahead of the frequency 
modulator.) We are concerned here with a quite different feature of 
PAM-FM. The piecewise constant transmitted frequency can be demod- 
ulated by means of circuitry which cannot handle the continuously 
varying frequency of the more usual FM signal. 

It is assumed that the demodulator is synchronized to the constant 
frequency intervals, as received. Some synchronization means are sug- 
gested in the Appendix (Section A.9). Then either correlators or matched 
filters may be used to estimate the piecewise constant frequency, sample- 
by-sample. The block diagram in Fig. 3 illustrates the concept, without 
filling in circuit details. A set of correlators or filters, tuned to a sequence 
of closely spaced frequencies, furnishes a spectrum analysis of the signal 
plus noise received over each sample interval. The signal is estimated by 
finding the frequency at which the spectrum is largest. 

The operation is complicated by the fact that the true frequency is 
anywhere in a continuum, and must be estimated to closer than 1 per 
cent of the bandwidth of the continuum. This implies something lke 
100 correlators or filters, or else means for interpolation which compare 
the outputs of adjacent units. 


2.1 A Spectrum Generator 


The set of correlators or filters furnishes an analog representation of 
the desired spectrum, in which positions along a sequence of output 
terminals correspond to discrete values of frequency. The techniques 
of radar pulse compression can be used to represent the same spectrum, 
with time as the analog of frequency, at a single output terminal. Ex- 
ternally, the circuit is equivalent to an infinity of correlators or filters, 
with scanning means to convert the spacially distributed outputs into 
a function of time. 

The spectrum generation hinges on a sequence of two operations. 
Fig. 4 is a block diagram. The first operation beats the received signal 
with a varying-frequency local oscillator, to obtain the difference fre- 
quency. Fig. 5 illustrates the frequencies of the true signal, of the local 
oscillator, and of the signal at the output of the mixer. The true fre- 
quency is constant over each sample interval, as before. The oscillator 
frequency varies periodically, in synchronism with the signal samples. 
In particular, it varies linearly over each sample interval. Thus, at the 


* For example, in telemetry systems. 
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Fig. 2 — Instantaneous signal frequencies. 


output of the mixer, the variations in frequency are the same over 
every sample interval, but the average varies from sample to sample. 

The second operation transmits the modified signal through a pulse 
compressing (dispersive) line. The nominal delay (phase slope) varies 
linearly with frequency. Over any one sample interval, the instantaneous 
frequency varies linearly with time. Thus the nominal delay varies 
linearly with time. Fig. 6 illustrates the variations in delay with fre- 
quency and time. 

The variations in delay are so scaled that the tail end of the signal 
sample just catches up with the head end. Then, on the basis of nominal 
delays, the entire signal sample emerges from the line in a single instant 
of time. Actually, of course, the nominal delay does not apply exactly 
to the time-varying instantaneous frequency. Thus the signal sample 
does not actually emerge from the line all at a single instant. However, 
under the conditions assumed it is squeezed into a small portion of the 
sample interval. 
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Fig. 3 — PAM-FM demodulation by correlators or filters. 
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Fig. 4 — Spectrum generation. 


The compression of a signal sample into a short pulse depends only 
on the variations in the instantaneous frequency, which are the same 
for each sample interval. On the other hand, the time of arrival at the 
output end of the line depends on the average frequency, which is the 
frequency of the true signal and varies from sample to sample. The 
baseband bandwidth and the FM index restrict the signal frequencies 
to a utilized RF bandwidth. With a suitable choice of circuit param- 
eters, the corresponding variations in arrival time cover a little less than 
one sample interval. Then the true signal produces one pulse per sample 
interval, whose position in a (somewhat delayed) sample interval is a 
measure of the signal frequency. Fig. 7 illustrates the situation. In 
other terms, beating with a swept frequency and then pulse compressing 
converts PAM-FM into pulse position modulation, or PPM.* 

It is now time to note specific formulas. For simplicity, let time ¢ be 
zero at the center of a typical signal sample interval. Let the true signal, 
for that interval only, be 


s(t) = ~/2P, cos (wt + 6s), . —-T/2<t< 1/2. (1) 


Here T is the length of the sample interval, w, and 8, are the frequency 
and phase of the true signal, and P, the signal power. Let the correspond- 
ing output of the mixer be 


$(t) = +/2P, cos (wd + Be — 3 at’), —~ T/2<t<T/2. (2) 


where q is an arbitrary constant. The instantaneous frequency is now 
w, — qt, linear with respect to time. [Actual circuitry may introduce 
constant changes in amplitude, carrier frequency and phase angle, 
between (1) and (2), but these are trivial for present purposes.] _ 


* In practice, the compressed pulse will have small side lobes, omitted in Fig. 7 
for simplicity. See Fig. 8 below and also Section A.9 of Appendix. 
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Fig. 5 — Frequency conversion. 
The corresponding output of the pulse compression line is approxi- 


mately 


S(t) = V/2P, F (w, — wr) cos (wet + 4 gt? + Bs — Be) 





Wr = w. + qt (3) 
gin M5 
F(A) = 


The expression assumes that w, is large compared with | w#, — w, | and 
| we — w. |. For present purposes, w, and «, lie in the utilized RF band, 
and w, is the midband, or carrier, frequency. A derivation of (3) from 
(2) is outlined in the Appendix (Section A.1). . 
The processed signal S(t) may be described as a high-frequency 
sinusoid multiplied by an envelope function. The frequency, ». + qt, 
varies with time, but it is independent of the received signal. On the 
other hand, the phase angle is 8, — 8. , in which 8, is a property of the 
transmission line, but 6, is the phase angle of the unprocessed signal 
s(t). The envelope is 1/2P, F(w, — wx). It is a function of time, but 
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Fig. 6 — Delay vs frequency and time. 


the time is an analog representation of the frequency variable w,. The 
signal frequency w, enters the envelope function as a parameter. 

Fig. 8 is a qualitative plot of F(w, — wx). The abscissae correspond 
simultaneously to time and w;. The largest F occurs at w, = w,. Thus 
the frequency w, may be determined by noting the time of the maxi- 
mum F’, and interpreting the time in terms of w, . The envelope F may 
be separated from the sinusoid by means of an envelope detector at 
the output of the line. Fig. 9(a) is a block diagram. 

For some purposes, it is convenient to divide S(t) into two com- 
ponents, as follows: 


S(t) = +~/2P, F(w. — wx) cos B, cos (wt + 4 gt? — B,) 


(4) 
— /2P, F(w. — ox) sin B, sin (wt + 4 gt? — B.). 
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Fig. 7 — Signals at terminals of dispersive line: 1, input signal, before pulse 
compression; 2, output signal, after pulse compression. 


The two sinusoids are independent of the signal s(t). Physically, the 
two envelope functions can be resolved by means of phase detectors. 
Fig. 9(b) is a block diagram. 

Consider the Fourier transform of a time function equal to s(t) in the 
one sample interval, and zero elsewhere. More specifically, consider the 
transform at positive frequencies w,; near w, . If the same approximations 
are made, as in the derivation of (3), the real and imaginary parts of 
the transform are the same as the two envelope functions in (4). The 
envelope function in (3) corresponds to the transform of the envelope 
of the original time function. 

The same remarks apply a little more generally. Suppose the ampli- 
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Fig. 8 — The function F(ws — wx). 
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Fig. 9 — Detection of envelope and components: (a) envelope, (b) components. 


tude of the received signal s(t) is modified, as well as the frequency, 
before it reaches the pulse compressor. Then §(¢) becomes 


S(t) = ~/2P, A(t) cos (wet + Be — 4 qt’), —T/2<t<T/2. (5) 


Suppose the envelope A(t) is symmetrical about the center of the sample 
interval. Then (3) and (4) apply except that F'(\) is now the transform 
of a time function equal to the new envelope during the sample interval, 
and again zero elsewhere. For the analogous radar application, see Ref. 2. 

The operations which convert (1) into (3) and (4) are all linear opera- 
tions on the signal. If s(é) is generalized to a sum of many constant- 
frequency sinusoids, the spectrum corresponding to a single sample 
interval can be generated by summing the results of the operations on 
the individual sinusoids. Referring again to the block diagrams, in Fig. 
9(a) the output is the amplitude of the transform, and in 9(b) the two 
outputs are the real and imaginary parts. We will use the collection of 
sinusoids as a representation of the signal plus noise, received during 
one sample interval. 

Thus pulse compression techniques generate analog representations 
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of the transforms of signal samples. The transforms are generated as 
functions of time. The constant q determines the time-vs-frequency 
scale, and can be chosen so that the utilized RF band is scanned in less 
than one sample interval. The width of the peak in Fig. 8 is merely the 
familiar “spectrum line width” of a sinusoid of finite duration. The de- 
tailed shape (in particular the tails) can be modified to some extent 
through initial multiplication by an envelope function (or, alternatively, 
by a shaping circuit at the output of the dispersive line). 

The same remarks apply to infinite sets of correlators or matched 
filters, except that the spectra are generated at specific instants as 
functions of position along arrays of output terminals. One result is: 
all three embodiments are equally sensitive to noise accompanying 
the received signal. A choice between the three must depend on practical 
compromises, limitations, etc., associated with the design of actual 
circuits. (For the external equivalence between correlators and matched 
filters, see, for example, Ref. 4.) 


III. SENSITIVITIES TO NOISE 


Demodulation by correlators, matched filters, or spectrum generators, 
as described in the previous section, will be referred to collectively as 
demodulation by spectrum analysis. This section compares the effects 
of noise in such circuits and in conventional FM receivers and FMFB. 
Between conventional FM and FMFB, some effects of noise are quite 
similar and some quite different. The two circuit forms will be referred 
to collectively as demodulation by frequency detection. 

It is assumed that the noise is Gaussian and that it is added to the 
signal before it reaches the demodulator. It may be, for example, thermal 
noise associated with first stages of amplification in the receiver. In 
demodulation by spectrum analysis, the noise adds random processes 
to the spectra analyzed. These may be described as two independent 
Gaussian processes added to the envelope functions in (4). The inde- 
pendent variable in the random processes is the spectral frequency 
wr , Which is also represented by time in the pulse compression embodi- 
ment. The processes are described in a little more detail in Section A.2. 

It is convenient to normalize the error formulas in terms of param- 
eters r, R, and T, defined as follows: 


w, = baseband bandwidth (0 frequency to cutoff) 

w, = full excursion of instantaneous signal frequency (maximum — 
minimum) 

r = w,/w, = bandwidth expansion ratio (6) 


350 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


_ P, = signal power ; (6) (cont.) 
P, = noise power in a frequency interval equal to one baseband 
= P./Pn = “signal power to noise density ratio” at the input of 


-. the demodulator 
_T = r/w, = baseband Nyquist interval. 


Under the conditions assumed here, the bandwidth expansion ratio, 
r, is fairly large — order of 10 or 20. Power thresholds (defined in the 
next section) set lower bounds on R, in the neighborhood of 14 to 16 db. 
In practical applications, practical compromises may require a somewhat 
larger R, and the bandwidth of the receiver must be a little greater than 
w, (whether demodulation is by spectrum analysis or frequency de- 
tection). The power spectrum of the noise is assumed to be uniform at 
the input of the demodulator, over the pertinent frequency interval. 


3.1 Two Different Effects of Noise 


For present purposes, one must examine two different effects of the 
noise, on the recovered baseband signal at the output of the demodula- 
tor. Under the conditions assumed here, the effects of the noise on the 
demodulated baseband signal are quite small most of the time. These 
may be called small noise errors, and their rms is one measure of circuit 
performance. On the other hand, during occasional brief intervals, 
peaks in the noise have a dominant effect and temporarily replace the 
true signal by a random false signal. This is commonly called blocking. 
It usually persists over intervals comparable with a baseband sample 
interval. The average number of blockings per second is the blocking 
frequency. . | 

Fig. 10 illustrates the two effects in terms of probability densities. 
It is a qualitative (not quantitative) plot of the probability density of 
the error, due to noise, in the demodulated baseband signal at any one 
instant. The peak near zero is substantially Gaussian and corresponds 
to the small noise errors. The long tails are flat and correspond to the 
probability that blocking will replace the true signal by a random 
signal. The transitions between the Gaussian peak and the flat tails are 
not considered further here. They are very difficult to calculate and 
must be strongly dependent on design details. 

The blocking frequency decreases very rapidly as the power ratio R 
increases. A related parameter is the power threshold. Thresholds of FM 
circuits (and also phase lock) have been defined in numerous ways for 
numerous purposes. The definition which best suits our present needs 
is the following: The power threshold is the signal power just sufficient 
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Fig. 10 — Qualitative form of error distribution. 


to meet a specified limit on the portion of the samples which are blocked. 
It can be expressed in terms of the corresponding ratio, R, in db. Under 
the conditions assumed here, the specified limit on the blocking rate 
may be perhaps one in a thousand or one in ten thousand. 


3.2 Small Noise Errors 


Consider first the demodulation of individual signal samples by spec- 
trum analysis. Both phase coherent and phase incoherent circuit forms 
are possible. More than one kind of phase coherence is of interest here. 
However, it will be simplest to start with the classical kind in which 
the phase of each constant frequency sample is independent of other 
samples and is determined uniquely by a rule known to the demodulator. 
This kind of phase coherence requires a degree of synchronization which 
may be impossible in practice.’ However, its theoretical properties bear 
on what follows. 

Under the conditions assumed here, the corresponding small noise 
errors are approximately as follows: 


For phase coherent demodulation: 


rms [small noise errors] _ 20/3 es 





max [true signal] a rR (7a) 
For phase incoherent demodulation: 
rms [small noise errors] _ 4/3 23 (7b) 


max [true signal] x rk 
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(The maximum true signal is here one half of a full signal excursion 
between equal -+ and — maxima.) Derivations are outlined in Section 
A.8. 

According to (7), the small noise errors of phase coherent spectrum 
analysis are about 6 db smaller than those of phase incoherent spectrum 
analysis, assuming that the phases of signal samples are determined 
individually and uniquely by a suitable rule known to the demodulator. 
How do these compare with the small noise errors of demodulation by 
frequency detection? 

Between conventional FM and FMFB, the small noise errors are 
approximately the same. More exactly, they are approximately the 
same functions of power level and bandwidths, which may themselves 
be quite different in practical applications of the two circuit forms. An 
approximate formula is 


For demodulation by frequency detection: 


rms [small noise errors] _ _2_ 1 (8) 
max [true signal] 4/3 rR° 


A well-known derivation is reviewed in Section A.4. 

Superficially, conventional FM and FMIB appear to be phase 
incoherent. However, the (theoretical) small noise errors are almost the 
same as in the phase coherent, sample-by-sample spectrum analysis. 
They differ only by a voltage ratio 7/3, or 0.40 db. This makes demodu- 
lation by frequency detection 5.62 db better, in regard to small noise 
errors, than the phase incoherent spectrum analysis.* It suggests that 
a more subtle form of phase coherence is at work, which perhaps can be 
realized also by a more subtle use of spectrum analysis. 

Further evidence is as follows: Consider the usual description of 
noise reduction by conventional FM demodulation. (See again Section 
A.4.) The frequency detector, as such, produces a demodulated baseband 
signal plus a substantial amount of noise. However, when the FM 
index is large, most of the noise power is at frequencies above the base- 
band. Fig. 11 illustrates the usual form of the power spectrum. Then a 
filter which passes only the baseband eliminates most of the noise. 

To approach the noise levels of phase coherent spectrum analysis, 
one must use an almost ideal baseband filter. But then the filter com- 
bines past outputs of the frequency detector over a “‘memory time” 
substantially longer than the baseband Nyquist interval. (Ideally it 


* The 6-db difference has been noted before, with different interpretation, by, 
for example, Kotel’nikov.® 
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Fig. 11 — Small noise errors in frequency detection. 


should be infinite.) Fig. 12(a) is a qualitative illustration of the appro- 
priate weight function, or impulse response. 

What happens if the filter is constrained to have a memory no longer 
than one baseband Nyquist interval? Suppose the true signal frequency 
is constant over that interval. Then the best weight function, within the 
constraint, is the parabola illustrated in Fig. 12(b).* The corresponding 
small noise errors turn out to be exactly as in phase incoherent spectrum 
analysis. _ 

It is not at once clear how the longer memory of the ideal, uncon- 
strained filter can reduce the (small noise) errors by anything like 5 or 
6 db. The original baseband signals are substantially uncorrelated over 
intervals longer than one Nyquist interval. The effective correlation 
time of the noise process is even shorter. However, it is the frequency 
of the FM signal which has the correlation characteristics of the base- 
band. The phase is further characterized by the continuzty of phase 
rotations required for a constant amplitude sinusoid of varying fre- 
quency. This may be regarded as a subtle kind of phase coherence 
which, in fact, is used effectively by the filter in demodulation by fre- 
quency detection. 

The interpretation is clarified and supported by the following argu- 
ment: Consider demodulation by spectrum analysis, and suppose the 
transmitted signal is generated by applying a piecewise constant control 
voltage to a frequency modulator. (See again Fig. 1.) Because the output 
of the modulator is a continuous sinusoid, the instantaneous phase 
rotation is continuous, even though its rate of change (which is the 
frequency) is discontinuous. The continuity of phase rotations, from 
sample to sample, has been called differential phase coherence. 


* “Parabolic smoothing’’ is best for a finite interval, and a constant signal 
plus noise power proportional to w?. See, for example, Ref. 6. 
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Fig. 12 — Filter weight functions: (a) ideal band-limiting filter; (b) optimum 
when constrained to one sample interval. 


Fig. 13(a) illustrates the differentially coherent phase rotations. The 
slope of each straight line segment is the frequency during one signal 
sample interval. In contrast, if the transmitted signal is differentially 
phase incoherent, the phase rotations are discontinuous between samples, 
as in Fig. 13(b). This corresponds, for example, to forming a piecewise 
constant frequency signal by successive selections (or keying) from a set 
of phase incoherent oscillators. 

Referring to Fig. 13(a), consider sample number k. The frequency 
can be estimated by an incoherent spectrum analysis of signal sample 
k by itself. [See again (7b) for the rms small noise errors.] Further 
information can be gleaned from spectrum analyses of samples k — 1 
and k + 1. Specifically, estimates can be obtained from these samples 
of the phase rotations at the beginning and end of sample interval k. 
Only the difference between the two phase angles is actually needed, 
and hence the absolute phase reference required for the phase coherence 
of (7a) is no longer necessary. 

The difference between the two estimated angles is the net phase 
rotation, modulo 27, over sample interval k. Dividing by the duration 
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(a) 


PHASE ROTATION —> 


(b) 


TIME -—> 


Fig. 138 — Phase rotations: (a) differential phase coherence; (b) differential 
phase incoherence. 


T of the sample interval gives a second estimate of the frequency, but 
only to modulo 27/7. When the noise is small, as assumed, the first 
estimate is accurate enough to resolve the ambiguity. Then a weighted 
sum of the two estimates gives an improved estimate of the true signal 
frequency. (The small noise errors in the two estimates are substantially 
uncorrelated.) Small further improvements can be derived from fre- 
quency and phase estimates for additional sample intervals. 

An optimum combination of phase and frequency measurements of 
all samples, —© to +, gives a 4.365-db theoretical improvement 
over sample-by-sample phase incoherent spectrum analysis. (The 
power ratio is 1 + ~+/3.) Of this, 3.979 db can be realized by using only 
samples k — 1, k, k + 1 to estimate the frequency of sample k. A deriva- 
tion is described very briefly in Section A.5. 

Why does one not realize the full 5.62 db apparent in conventional 
FM demodulation? It can be interpreted as a curious effect of the sam- 
pling of the original baseband signal, which is not part of the conven- 
tional FM system. The interpretation is supported by what follows. 

Suppose the piecewise constant frequency is applied to the frequency 


356 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


detector in an (idealized) conventional I'M receiver, and that the noise 
level is low enough to justify the usual small noise approximations. The 
output is a piecewise constant true signal, like curve A of Fig. 14, plus 
noise. The noise can be reduced by sampling the output of a suitable 
filter, as suggested by Fig. 15. Can the ideal baseband filter be used, as 
for an unsampled signal? 

Elementary information theory includes the following: If the samples 
were represented by a sequence of very short impulses, like curve B of 
Fig. 14, the ideal filter would be as effective as for the unsampled signal. 
However, because they are represented, in fact, by a piecewise constant 
signal, like curve A, the ideal filter has two shortcomings. It produces 
intersample interference. It responds to the wanted sample less effi- 
ciently than to an ideal impulse. 

Suppose the filter is constrained to give no intersample interference, 
assuming each sample to be a constant signal over its entire sample 
interval. The best filter within the constraint gives 4.365 db improve- 
ment over incoherent sample-by-sample spectrum analysis, which is 


A — PIECEWISE CONSTANT SIGNAL 
B — SEQUENCE OF IMPULSES 


SIGNAL —> 





TIME —> 


Fig. 14 — Filter inputs. 
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PIECEWISE 
PAM-FM FREQUENCY CONSTANT SIGNAL 
DETECTOR FILTER SAMPLER 


Fig. 15 — Filter and sampler after frequency detector. 


exactly the same as the figure for multisample spectrum analysis using 
differential phase coherence. A derivation is outlined in Section A.6. 


3.3 Thresholds 


Consider first the thresholds of sample-by-sample spectrum analysis. 
Fig. 16(a) illustrates the spectrum of the usual signal-plus-noise sample. 
Fig. 16(b) illustrates the spectrum of the occasional sample which blocks. 
It assumes that the frequency of the spectral maximum is used as the 
estimate of the true frequency, as before. The blocking occurs when the 
spectrum of the noise sample has a peak, at a random frequency, which 


Oo 





SPECTRUM =——> 





SPECTRAL FREQUENCY, Wy, =— > 


Fig. 16 — Spectrum of a single signal-plus-noise sample: (a) the usual sample; 
(b) the occasional sample which blocks. 
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exceeds the spectrum of signal-plus-noise at the true signal frequency. 
The remarks apply to both phase coherent and phase incoherent sample- 
by-sample spectrum analysis, provided the pertinent spectra are used 
for each. 

The corresponding blocking probabilities are approximately as follows: 


For phase coherent spectrum analysis: 
ee! 2 
P= aJaR exp (—R*/4). (9a) 


For phase incoherent spectrum analysts: 


r—2 


P= exp (—R’/4). (9b) 





Here P is the probability that a typical sample is blocked, and blocking 
of different samples is uncorrelated. 

Part of the derivation is the same as for the (gross) error rates of 
quantized frequency shift keying (FSK). However, there is an. extra 
complication. In FSK, one is interested only in the spectrum at a finite 
set of discrete frequencies. The random process which is the noise 
spectrum is at most weakly correlated between the pertinent frequencies. 
Thus error rates have been approximated, for example, by assuming 
either zero correlation’ * or a manageably simple form of correlation.® 

For our purposes, we must consider the spectra at all frequencies in 
a continuum, with the certainty that correlations are high across small 
frequency differences. An exact calculation would be extremely difficult. 
As an approximation, one can proceed as follows: Divide the pertinent 
frequency interval into, say 7 equal subintervals. Approximate the true 
spectrum in each subinterval by a constant. Assume that the constants 
for the 7 subintervals are independent random variables (over the 
ensemble of noise samples). Now one can estimate blocking probabilities 
as error rates in an y-frequency FSK system. Differences between (9) 
and equations in Refs. 7 and 8 reflect further approximations, appro- 
priate under the conditions assumed here. They are described briefly 
in Section A.7, together with some further analytical details. 

The approximation to the spectrum may be described further as 
follows: The covariance of the spectrum of the noise sample is approxi- 
mated by perfect correlation over each subinterval and zero correlation 
between subintervals. The actual correlation across the (radian) fre- 
quency difference w. — w is 
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sin (wo — w) = 
a (10) 
G=e 
2 1 9 


(see Section A.2). Equations (9) correspond to subintervals of width 
2w», which is the | w: — w:| at the first zeros of the true covariance 
function. 

We have defined the threshold as the signal power required to meet 
a specified limit on the blocking frequency. The corresponding power 
ratio &, used in (9), must give the single-sample blocking probability P 
which corresponds to the specified blocking frequency. 

Under the conditions assumed here, P is very small, say 0.001 or 
0.0001. Then the exponentials in (9) are very small, and small per- 
centage changes in R produce much larger percentage changes in P. 
As a result, changes in the coefficients, multiplying the exponentials, 
can be compensated by much smaller changes in R. For example, a 
two-to-one change in a coefficient is offset by something like a $-db 
change in R. Two consequences are as follows: The threshold changes 
only slowly with the bandwidth expansion ratio r. The threshold is 
rather insensitive to the size of the frequency subintervals used in the 
approximation described above. 

Numerical examples of thresholds will be tabulated in Section IV, 
together with small noise errors. 

Slepian’? has derived from general information theory some important 
upper and lower bounds on the thresholds (as here defined) of quantized 
systems, constrained to code baseband samples individually, for trans- 
mission over channels wider than the baseband. It is interesting to 
compare the thresholds (9) with Slepian’s bounds, even though (9) 
refers to unquantized systems. Since the bounds depend on the number 
of quanta, one must first decide on the appropriate quantization. 

Transmission and demodulation of a quantized signal, as such, in- 
volve no counterpart of the small noise errors in unquantized systems. 
However, when the original baseband signal is unquantized, transmis- 
sion in quantized form implies quantization or round-off errors relative 
to the original signal. Then, in judging system quality, one can compare 
the quantization errors in a quantized system with the small noise errors 
in an unquantized system. Thus it is interesting to compare thresholds 
determined by (9) with Slepian’s bounds for quantized systems such 
that the rms quantization errors match our rms small noise errors. 

Our present purposes are served by a very rough comparison, using 
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graphical data in Slepian’s paper. Under the conditions assumed here, 
the thresholds (9) are only very little above Slepian’s lower bound. The 
differences are very roughly + db for sample-by-sample phase coherent 
demodulation and one db for the phase incoherent form. 

In principle, the thresholds can be reduced even a little further by 
combining phase and frequency estimates derived from more than one 
sample interval. We have seen that a second estimate of the frequency 
of sample k can be derived from the phases of samples k — 1 and k + 1. 
The same is true of the phase of sample k. This permits the phase 
coherent threshold to be approximated with only differential phase 
coherence. More complicated operations yield a further improvement. 
Referring again to Fig. 16(b), blocking occurs when a noise peak exceeds 
the signal peak, in the spectrum of the signal plus noise, and is chosen 
in its place. The additional phase information can be used to improve 
the choice between the two peaks. However, the 27 phase redundancy 
severely limits the improvement. For the conditions assumed here, a 
rough estimate is a ten-to-one reduction in the blocking frequency, or 
something like a one-db reduction in the threshold at the old rate (rela- 
tive to phase incoherent spectrum analysis). A few further details are 
noted in Section A.8. 

The improved threshold may be slightly below Slepian’s lower bound. 
This is not improper, since it is obtained by violating Slepian’s assump- 
tion of sample-by-sample coding and decoding. 

Now consider the thresholds of conventional FM demodulators and 
FMFB. Fig. 17 compares simplified block diagrams of the two circuit 


WIDE-BAND FREQUENCY BASEBAND 
— FILTER DETECTOR FILTER —> 
(a) 
NARROW-BAND FREQUENCY 
FILTER DETECTOR 
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OSCILLATOR 







BASEBAND 
FILTER 









(b) 


Fig. 17 — Demodulators using frequency vere eunn (a) conventional FM 
demodulator: (b) demodulator using FM feedbac 


FM DEMODULATION 361 


forms. The blocking phenomenon is a well-known characteristic of 
these circuits. Under the conditions assumed here, the thresholds are 
significantly lower (permit lower signal power) in FMFB circuits than 
in conventional FM receivers. The advantage derives from the relative 
bandwidths of the filters just ahead of the frequency detectors, and 
thereby depends on a fairly large bandwidth expansion ratio, r (which 
is here 10 or 20). This is, of course, the reason why FMFB is of current 
interest, for example for satellite communication systems." 

Because of the nonlinear feedback loop, it is extremely difficult to 
calculate for FMIFB the quantitative thresholds required for specific 
blocking rates. However, important parameters have been identified 
and studied, for example by Enloe.” Good circuits have been built and 
demonstrated for voice and television channels, with thresholds which 
are not far above the theoretical lower bounds. Since the quantitative 
blocking rates have not been determined, the margins above the bounds 
are not known exactly. 


3.4 Comparisons with Other Methods 


At noise levels and blocking rates appropriate for television, telephone, 
and carrier telephone, FMFB and spectrum analysis of PAM-F'M have 
lower theoretical thresholds than binary PCM. The binary symbols 
are less sensitive to noise than, say, PAM-IF'M samples received at the 
same rate. If this were the whole story, binary phase modulation would 
have the smaller threshold by a power ratio of about two.* Actually, 
of course, the symbol rate must be greater than the baseband sample 
rate by a factor, say p, equal to the number of binary symbols per 
sample. This, in itself, raises the power threshold by factor p. Thus, if 
there are more than two symbols per sample, the theoretical threshold 
for binary phase modulation is larger, by a power ratio of about p/2. 

The threshold ratios are about the same if one compares the binary 
PCM with the following FSK system: A set of, say, 10 discrete fre- 
quencies is used, spaced orthogonally in the usual signal theory sense. 
, One frequency from the set is transmitted during each baseband sample 
interval. But this system has only 10 quantum levels. To obtain, say, 
100 quantum levels one must either transmit two symbol intervals per 
sample (which raises the threshold 3 db), increase the channel bandwidth 
by a factor of 10, or pack the frequencies much more closely than the 
orthogonal spacing. With close spacing, errors of one quantum level 


_ * Binary phase modulation requires less power than binary frequency modula- 
tion. See, for example, Sunde," 
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are more probable than larger errors, and there comes a point where they 
are more like the small noise errors of the analog systems. 

In principle (but not likely in practice) thresholds can be reduced 
by using systems with fewer symbols or samples per second than the 
baseband sample frequency. For example, two baseband samples can 
be transmitted as a single analog sample provided the signal-to-noise 
ratio can be doubled (>80 db instead of >40 db). Transmission at the 
reduced sample rate yields a small reduction in threshold. It is paid for 
by an enormous increase in the channel bandwidth, which is required 
for the higher signal-to-noise ratio. 

If more and more samples are combined, Shannon’s fundamental 
channel capacity is undoubtedly approached. Turin“ and Golay1 
have demonstrated that two closely related systems do, in fact, approach 
the theoretical capacity.* 

Our formulas for demodulation by spectrum analysis assume that the 
true signal is estimated by finding the maximum point in the pertinent 
spectrum. The same is true of the analysis of FSK error rates in Refs. 
7, 8 and 9. A well-known substitute for the determination of a maxi- 
mum uses a circuit whose output is zero except when a signal-plus-noise 
(in this case the spectrum) exceeds a preset threshold. The threshold is. 
set so that, most of the time, the peak due to the true signal and only 
that peak gets through. 

Under the conditions assumed here, the threshold circuit form in- 
creases the theoretical power threshold by very roughly 3 db. More 
exactly, the blocking probability is dominated by an exponential factor 
exp (—R?/8) as opposed to exp (—R?/4) in equations (9). 


IV. CONCLUSIONS 


The techniques of radar pulse compression can be used to generate 
spectra of signal samples as analog functions of time. It can be done in 
real time in the sense that the spectrum of each signal sample is scanned 
in a time no greater than the sample interval. The spectra are the same 
as would be generated by infinite sets of correlators or matched filters. 
Spectrum generation of this sort may be useful for various purposes, 
particularly where the parameter ranges are suitable for the sort of 
hardware which has been developed for radar pulse compression. 

Demodulation by frequency detection (with or without feedback) 
reduces the small noise errors by a disguised but efficient use of differen- 

* The increase in channel bandwidth as Shannon’s limit is approached is merely 


a property of these specific modulation schemes. In principle, it is necessary only 
to increase the length of the pieces of the signal which are coded as units. 
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tial phase coherence, which is a characteristic of FM signals. Demodula- 
tion by spectrum analysis can also take advantage of the differential 
phase coherence, although the pertinent operations are fairly compli- 
cated. The piecewise constant signal frequency, needed for the spectrum 
analysis, reduces the effectiveness by 1.24 db in the theoretical small 
noise errors (which can be offset by a 15 per cent increase in the FM 
index). 

Under the conditions assumed, and for thresholds as defined here, the 
theoretical power thresholds of the spectrum analysis are very close to 
Slepian’s lower bound. The power threshold of FMFB appears to be 
quite close, but just how close has not been determined. 

Thus, under conditions appropriate for television, telephone, and 
carrier telephone systems, the theoretical noise sensitivities are very 
little different in FMFB and in PAM-FM with demodulation by spec- 
trum analysis. Both techniques pose numerous practical problems, 
relating to, for example, stability requirements, switching time require- 
ments, synchronization to signal samples, over-all complexity, non- 
linearity in response to true signal, etc. FMI'B has the advantage that it 
has already been used, although under somewhat special conditions. 

Some theoretical thresholds and small noise errors are collected in 
Tables I and II, for various blocking probabilities P and bandwidth 
ratios r. They were calculated by (7) and (9) and refer to demodulation 
of PAM-FM by phase coherent and incoherent, sample-by-sample 
spectrum analysis. A few remarks on circuit problems are collected in 
Section A.9. 

The noise figures obtainable with practical circuits are of course 
somewhat poorer. The degradations may be due to rather different 
practical compromises in circuits using spectrum analysis and in FMFB. 
Comparisons between practical noise figures may be different for differ- 
ent applications. 

Under some conditions, a combination of spectrum analysis and 
frequency detection may be preferable to either alone. Fig. 18 is a block 
diagram of one out of many possible arranagements. A spectrum ana- 
lyzer furnishes a first estimate of the frequency of a PAM-F™M signal, 
using phase incoherent, sample-by-sample spectrum analysis. The 
estimated frequency variations are generated locally by a voltage- 
controlled oscillator. A mixer subtracts the oscillator frequency from 
the frequency of the received signal. (The block labeled “‘delay” allows 
for the operation time of the spectrum analysis.) Then the output of the 
mixer is very low index FM, corresponding to the errors in the first 
frequency estimate, plus noise. 


364 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


TABLE I — THRESHOLDS AND SIGNAL-TO-NOISE RATIO 
FoR PHASE COHERENT SPECTRUM ANALYSIS 

















Probability of Blocking Bandwidth Ratio Threshold [== Demod, Signal"]* 
P 1 = w/o Ratio Ps/Pr rms Small Errors 

(db). (db) 
0.01 . 10 12.1 © 81.3 
0.005 10 12.7 31.9 
0.002 10 13.4 32.6 
0.001 10 13.9 33.0 
0.0005 10 14.3 33.5 
0.0002 10 14.8 33.9 
0.0001 10 15.2 34.5 
0.01 20 12.8 38.0 
0.005 20 13.4 38.5 
0.002 20 14.0 39.1 
0.001 20 14.4 39.5 
0.0005 20 14.8 39.9 
0.0002 20 15.3 40.4 
0.0001 20 15.6 40.7 
0.01 40 13.4 44.6 
0.005 40 13.9 45.0 
0.002 40 14.4 45.6 
0.001 40 14.8 46.0 
0.0005 40 15.2 46.3 
0.0002 40 15.6 46.8 
0.0001 40 15.9 47.1 


* At threshold signal power. 


Because of the low index, it is now appropriate to use a narrow-band 
filter (passing something over two baseband bandwidths) followed by a 
frequency detector and a low-pass filter. The sampled output of the 
filter furnishes a correction to the first frequency estimate. The theoreti- 
cal threshold of the combination is the same as for phase incoherent 
spectrum analysis. The theoretical small noise errors are the same as 
for demodulation of PAM-I*M by frequency detection. The theoretical 
improvement over the small noise errors of the first frequency estimate 
is 4.3865 db. 

If the spectrum analysis is accomplished by correlators or matched 
filters, a moderate number may be sufficient even though the over-all 
errors must be >40 db below the true signal. The error determination 
by frequency detection can correct for a fairly coarse quantization of 
the first estimate at the same time that it reduces the errors due to noise. 

The over-all circuit may be described as open-loop tuning to the pass- 
band of the narrow-band filter, as opposed to closed-loop tuning in 
FMFB. 
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TasBLE II — THRESHOLDS AND SIGNAL-TO-NoIsE RATIOS 
FoR PuasE INCOHERENT SPECTRUM ANALYSIS 


Probability of Blocking Bandwidth Ratio Threshold Max. Demod. Signal"]* 
P r = w/w Ratio Ps/Pn, rms Small Errors 

(db) (db) 
0.01 ‘10 13.3 26.5 
0.005 10 13.8 27.1 
0.002 10 14.4 27.6 
0.001 10 14.8 27.9 
0.0005 10 15.2 28.4 
0.0002 10 15.7 28.8 
0.0001 10 16.0 29.3 
0.01 20 13.9 33.1 
0.005 20 14.3 33.5 
0.002 20 14.9 34.0 
0.001 20 15.3 34.4 
0.0005 20 15.6 34.7 
0.0002 20 16.0 35.1 
0.0001 20 16.3 35.4 
0.01 40 14.4 39.6 
0.005 40 14.8 40.0 
0.002 40 15.3 40.5 
0.001 40 15.6 40.8 
0.0055 40 16.0 41.1 
0.0002 40 16.3 41.5 
0.0001 40 16.6 41.8 


* At threshold signal power. 
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Fig. 18 — A combination of spectrum analysis and frequency detection. 


SAMPLER 


BASEBAND 
SAMPLES 








ANALOG 
BASEBAND 
SIGNAL 














366 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


Vv. ACKNOWLEDGMENTS 


The author’s work, reported here, has been strongly influenced by 
conversations with numerous colleagues in the data transmission, FM 
and FMFB, information theory, and radar pulse compression fields. 
These included V. E. Benes, W. R. Bennett, L. H. Enloe, E. N. Gilbert, 
W. M. Goodall, C. W. Hoover, Jr., I. Jacobs, R. A. Kaenel, D. Mitchell, 
S. O. Rice, C. L. Ruthroff, and D. Slepian at Bell Telephone Labora- 
tories, and T. Kailath, R. Price, and G. L. Turin while visiting the 
University of California at Berkeley. Several of the references were 
supplied by reviewers. 


APPENDIX 


A.1 Spectrum Generation by Pulse Compression 
Tor a signal sample, modified by the local oscillator, assume: 
§(t) = V/2P, E(t) cos (wt — 5 qt’ + Bs) 
E(t) = 0 outside of interval —T/2 St S$ +T7/2 
E(—t) = K(t). 
Yor the impulse response of the pulse compression line, assume: 
w(t) = cos (wet + 4 qt” — Be). 


When |» — w, | K w,, the frequency function is 
2q : 
The output of the line is §*w. Integrate only over H(t) 4 0: 


sao +7/2 
S(t) = VIP. [__ B(r) cos (wr — Bar? + 8.) 


=—_— 


-cos [w(t — r) + 4 q(t — r)? — Bl dr. 





Express the integrand as a sum of cosines. Neglect the high-frequency 
term. Then: 
+7] 


2 
S(t) = V3P, [_ BE(r) cos lot + 4 al +B. — 6. 


+ (we — we — gt)r] dr. 
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Resolve into components per sin, cos [(w, — w. — gt) 7]. 
Recall that Lr) is even. Then £(7) sin [(w; — w. — qt) 7] is odd in +. 


S(t) = /2P, F(w, — w — gt) cos (wt + 3 qt? + Bs — Be) 
+7/2 
F(d) =| LEG doe- On) dk 
T/2 


t— 


sin \ 5 
When #£(7r) = 1, -T/2 S 7 S 4+T7/2, F(A) = , 





A.2 Noise Contributions to Observed Spectrum 


Following Rice,” but sacrificing some details of rigor to brevity, let 
the noise at the demodulator input be: 


Bin [ Dre Cre ea eee - HOV er ee re 


The interval w, to we includes all signal frequencies w, . 

Phase 8, = an arbitrary parameter in noise representation. 

z(w), y(w) = uncorrelated, zero average random variables, with 
uniform variances, and zero autocorrelations except across infinitesimal 


frequency intervals. 
Let Ave denote an ensemble average, or expectation. 
Let w:(w) and we(w) be arbitrary, except for the pertinent conditions 


of integrability. 


ie {[ ew wy(«) de [2 ne ish a, - a) was 
ee Pv eas [- wle) iB) ish ae [- odds 


Ave i (a) wade i. iS) a) ash = 


. . : 2 
Py, = noise power in one base bandwidth = wo". 


Let'N(w,) = the noise part of the spectrum of one signal-plus-noise 


sample. ea 
Apply Section A.1, with w, = w and w, + qt = w,, to integrands in 


n(t). 
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N(t) = NiCr) cos (wet + 4 — Be) 
+ N2(wx) sin (wt + 34° — Be) 


WG) = [ aa eas 


N2(wr) = i y(w) Fw — wr) dw 


Ni(wr), Ne(wx) = independent Gaussian random processes in wz . 


Appropriate choices of w; , wz in the above expectation integrals give 


autocovariances of Ni, Ne. 





2 
Ave [N (wz) Ny(w;)] = o | F(a — wr) F(w — w;) de, y = 1,2. 
@4 
Approximate the integration by integrating from — © to +o. 
Refer to Section A.1 and use E(t) = 1, -—T/2 Sts 4+77/2. 
sin (w, — w;) S 
Ave [N,(wz) N_(@;)] = xo” —~———_——~ . 
(co — w;) 
Po Ps pe es 
We 


Let w; = wp, refer to (3), and recall that R’ = , 
P b Wpe 2 


Max of Signal Spectrum _ / P, _ = 
rms Ny(wz) = P, I, saa 
A.8 Small Noise Errors in Sample-by-Sample Spectrum Analysis 
Refer to Fig. 4, S(t) of (3), and N(wz) of Section A.2. 
Use on = w + gtand B, = B;. 
S(t) + N(t) = [V2P. F(ws — on) + Ni(ox)] 


X cos (wt + £ gt’ + Bs — Be) 
+ No(wx) sin (wt + 4 gt? + B — B.). 


Assume (for small noise errors only): 
NY, Ne «& 2P.F°(0), 
Phase Incoherent Spectrum Analysis. Neglecting N.’, the envelope is 


é <K wy. 


Wr — Ws = €, 


/2P, F (as = wp ) — Ni(arx). 


Form a power series in ¢ and solve for max with ¢ small. 
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2 
Ave (2) 
Aig = OE, 


oP, (24 2) 





0 e e=0 


Evaluate by (3) and Section A.2 to get (7b). 
Phase Coherent Sample-by-Sample Spectrum Analysis. Refer to (1). 
Make the phase £, a linear function of o, : 


So = /2P, cos [wt + (w, — w,)(T'/2)]. 


Find the components of S(t) and N(é) in phase with a locally 
generated cos {[w, + qg(7/2)|t + 4 qt’ — B.}. 
Refer to (3). Let S, be the component of S. 


S.(t) = VW2P, F(ws — wx) cos [(ws — ox) (T/2)] 
= \/ 2P, F[2(w, — wz. 


It can be shown that the frequency variable is also doubled between 
covariances of Ni(w,) and its counterpart here. Hence noise is accounted 
for with 4 the frequency errors e. 


If the frequency-dependent signal phase appears artificial, change 
the time scale tof = ¢ + 77/2. 


S(£) = VP, cos (ad — Fen), C= f= 7. 


A.4 Small Notse Errors in Frequency Detection 


The FM signal is now unsampled. For simplicity assume a constant 
signal frequency. Resolve the noise per signal phase. 


s(t) + n(t) = [\/2P, + na(t)] cos (wt + Bs) 
+ m(t) sin (wt + Bs) 





_ N(t) 
s(t) + n(t) = pcos [wt +B, + ¢(t)], tang = ap. + nit)’ 
The unfiltered frequency error is ¢. Refer to Section A.2 to get: 
= Ave ny _ o =e 
POP OP Iai 


The ideal baseband filter passes only | w — w,| S wo. 





When n’? K 2P,, Aveg (w — w,)” dw. 
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ee none | aay = Zoe worPy _ on! 
{ = Ss a eS ESS 
ve (Tiltered «) SP. |, dr SP, 3P, SR 


A.& Small Noise Errors in Multisample Spectrum Analysts 


Refer to (1) and Fig. 13(a). Let (w., 6.) = w, and the midsample 
phase 6, of sample oc. With differential phase coherence, 


Bo — Boa = (We + W@o1)(T/2). 


Let n,m, = noise contributions to observed w, , (2/T) B.. 
Define x, , y. , 2, and note the relation to errors: 


te = We FE Me, Yo = (2/T)B. + me. 
fg = (a + Lo-1) = (Yo =, Yo-1) = (no + No—-1) = (me = Mo-1). 
Let w, + ¢ = the following estimate of w, : 
-+00 
We fte= 4% — D> Q) jz; . 
j=—oo 


2 2 2 
Let on = Aven, om = Avem, 


Ave e = [ — 2(Q. + Qe) + > (Q; + Qi4)"] on 
+ [ > (Q; - Qiss)”] Sin: 


Choose the Q;’s for min. Ave ¢’ by the calculus of variations. 
Compare with Ave ¢ for x, alone, which is o,’. 


: 2 
Min Avee ofsum om 


Ave é of z, alone on + om 
Further analysis like that of Sections A.2 and A.3 gives 
2 2 
On = 3m 


‘ 2 
Min Ave € of sum _ 


1 
Ave 2 of x, alone 1+ +3 One ee: 


A.6 Small Notse Errors in Multisample Frequency Detection of PAM-FM 


Refer to Section A.4 but assume only a piecewise constant signal 
frequency. Refer to Figs. 13(a) and (15). 

Let w(t) = filter weight factor, referred to the output sample time. 

Assume w(+o) = 0. 
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+00 +00 
Mie arer = A _ w(t) eat = - i w(t) o(L)dl. 


Use g(t) = [n(t)/+/ 2P,] and a white noise approximation. 





A 2 a” bs : 2 
vee = \/ oP. - [w(t)] “at. 


Find w(t), which gives 

(a) normalized response to constant frequency in sample oc, 

(b) zero response to constant frequencies in samples other than o, 
(c) minimum Ave ¢ within constraints a, b. 


The calculus of variations makes w(t) quadratic over each sample 
interval and continuous at the boundaries. Then Ave ¢ is a quadratic 
sum of the boundary values. Minimizing the boundary values is like 
minimizing the coefficients Q; in Section A.5 (with co,’ = 30) and 
gives the same result. 


A.7 Blocking Probability in Sample-by-Sample Spectrum Analysis 


Refer to Sections 3.8 and A.38. Approximate Ni(wz), Ne(wr) by 
processes piecewise constant over 7 subintervals. 

Approximate +/2P, F(w; — wr) by ~/2P, F(0) over the subinterval 
s and zero elsewhere. 

Let 2 , yy. = the components of the signal-plus-noise spectrum, scaled 
(normalized) to unit variances. The probability densities are: 








—_ 1 1 — BY te 
D, = x exp | aa emai 
2 
_ + ry + Yr 
KY exp ( 2 ), Nx 8. 


Phase Coherent Sample-by-Sample Spectrum Analysis. Rotation of 
the x, , 7, axes through 7/4 gives quickly 


P{m > 2, |d ¥ 8} =3[2 ~ mit (5), 


Exf (r) = (2 [ ex (- “) Te 


This is the probability of a specific 1, > x, , out of 7 — 1 m’s,rA ¥ 8. 
Under the conditions assumed here, the probability of any one or 
more Is: 
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me (n — 1)P{am > %} = 5+ E ce (7a) | 








Per Section 3.3, use 


Phase Incoherent Sample-by-Sample Spectrum Analysis 


2 
P{(ay’ + yn) > (20 + ys')} = 5 exp (-*). 


Under the conditions assumed here, for one or more 2’s, \ # s: 


7—1 





Pee 


R? 
exp (-") ; use n = 1r/2 as before. 


The last approximation is here a simplification, not a necessity. lor 
an exact formula (given D, , D, as above) see Ref. 7 or 8. 
A.8 Reduction of the Blocking Rate of Spectrum Analysis 


Refer to Section A.5. Use x, y of A.5. For a second estimation of a, , 
(1/T) (Bs41 — Bsa) — ¥ (@s41 + @s-1) 
We + € = § (Yet — You) — § (Sei + Vea) + (2vr/T) 

vy = unknown integer due to phase ambiguities. 


Refer to Fig. 16(b). Find the integers » for the best fits to frequencies 
of the two peaks in the signal-plus-noise spectrum. 

With no weighting for the heights of peaks, the probability that the 
closest is the correct choice is of the order of 0.9 (under the system 
conditions assumed here). 


ll 


Ws 


The actual choice must use also the relative heights of the peaks. 

Let Pu(M,, M,) = the probability density of the maxima M,, M, 
at the peaks due to signal-plus-noise and noise only (respectively). 

Let P.(é€s, €r) = the probability density of the observed deviations 
és, €n Of the second w, from the location of the peaks, using best »’s. 

Use subscripts 1, 2 for the M’s and e’s before the identification of 
which peak is signal-plus-noise and which is noise only. 

The best identification corresponds to the larger of 


Pxu(M,, M2) Pa, €) and Pu(M, ) M,) Pie, €1). 
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P gives a strong weighting except when M,; is close to M1. 
But when M, > M,, the difference is usually small, and P, only 
very rarely gives a strong weighting to a wrong choice. 
tag (u) Ru 


Let u = M, — M,. Then P(—u) = 6 





A complete calculation of the probability of a correct choice would 
require integration over a complicated portion of the 4-dimensional 
space of M,, Mn, €, €n. 


A.9 Some Circuit Considerations 


A few circuit considerations are described below in brief, purely 
qualitative, terms. 

Synchronization of Spectrum Analysis to PAM-FM Samples. Assume 
the following: The spectra represent signal-plus-noise received during 
intervals locally selected by a precision oscillator or clock. The length 
T of the intervals is almost right, without synchronizing means. The 
problem is to synchronize the start time to the start times of the true 
signal samples. 

Synchronizing signals might be obtained by any of several means. 
One uses a very narrow band transmission channel, to send synchroniz- 
ing signals from the transmitter. Others derive synchronization error 
signals from the communication signal itself, which must fluctuate 
sufficiently to supply the necessary information. (When the true signal 
is constant from sample to sample, there is nothing to indicate the 
boundaries between samples.) An error in synchronization reduces the 
height of the peak in the signal spectrum (on the average). It also pro- 
duces a discrepancy between values of w, obtained from the single 
sample spectrum and by the second method described in Section A.5. 
In principle at least, a synchronization error signal can be derived from 
either effect and can be averaged over many sample intervals to reduce 
the effects of noise on the synchronization. 

Shape of the Signal Sample. In (8), the tails of the function F are 
neither small nor short. By Section A.1, they can be reduced by shaping 
the envelope E(t) of the signal sample before forming its spectrum. A 
suitable filter in the output of the spectrum generator has the same 
effect. Since the best spectral maximum corresponds to the F of (8), a 
practical compromise is needed. The pulse shaping problem is an old 
one, but here intersample interference due to the tails is not the im- 
portant problem, but rather the way the tails can increase the blocking 
probability (noise-plus-tails exceeding signal-plus-noise). 
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Channel Bandwidth. For both ordinary FM and PAM-F'M the channel 
bandwidth must be a little wider than the full excursion, w, , of the 
instantaneous signal frequency w,. The so-called Carson’s Rule calls 
for a channel width of w, + 2w, for ordinary FM, and the appropriate 
rule for PAM-FM is at least not very different. FMFB and PAM-FM 
spectrum analysis can tolerate wider bands without significant changes 
in thresholds and small noise errors. 

Transition Intervals. In idealized models of spectrum analysis, certain 
operations happen in zero time. In any actual circuits there will be 
nonzero switching times. Very roughly, if a fraction a of each sample 
interval is lost due to the switching times, the signal power must be 
increased by factor 1/(1 — a). Thus 2 per cent lost time requires roughly 
0.1 db more power. In a sense, switching times are spectrum analysis 
counterparts of feedback stability problems in FMFB, although the 
comparison is purely qualitative. 
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Data Transmission over a Self-Contained 
Error Detection and Retransmission 


Channel 


By F. E. FROEHLICH and R. R. ANDERSON 
(Manuscript received Deeember 10, 1962) 


Error control of the detection and retransmission type requires an internal 
storage buffer when the data source cannot be stopped. With finite capacity 
there will be occasions when this internal buffer ts overfilled. This paper 
investigates the relationships among the error statistics of the channel, the 
storage capacity of the buffer, the round-trip transmission delay and the bit 
rate from the source. It is shown that the process can be treated as a Markov 
chain. The solution algorithm is programmed for machine computation, 
and representative cases are solved numerically. For typical values selected 
from the telephone plant, it is found that buffer capacities of a few hundred 
bits would be adequate. 

The technique described should be useful for solving other problems in 
queueing theory. 


I. INTRODUCTION 


Studies during the last few years have shown that in the transmission 
of digital data over telephone lines, high accuracy can be achieved when 
the message is encoded in an error detecting code. Correction can then 
be accomplished by a repeat transmission of the portion of the informa- 
tion containing the errors. These so-called ‘‘feedback”’ techniques have 
been shown to be very effective in controlling errors.!:?:34 

For some sources of data it is inconvenient or impossible to have the 
source wait while previous data are being retransmitted. There are also 
cases where it is required that the output from the receiver be at a 
uniform rate. This memorandum describes a self-contained error detec- 
tion and retransmission channel capable of accepting data from the 
source at a steady rate, or at any rate less than a specified maximum, and 
of delivering it to the sink at this same rate. The channel is “self-con- 
tained,’’ meaning that the channel itself provides enough storage of in- 
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formation to permit the detection of errors and their correction by re- 
transmission without the data source and sink being aware that these 
processes are going on. The data source merely puts data into the trans- 
mission system at its own rate, and the data sink accepts highly reliable 
data from the system at the same rate. The relationships among system 
delay time, error probability, bit rate, and storage capacity are investi- 
gated. 

The use of feedback error control with a data source which cannot be 
interrupted was briefly discussed by Reiffen, Schmidt, and Yudkin.® A. 
B. Fontaine has simulated such a system on a computer, using error data 
collected on private wire circuits.* Our analysis has indicated that shorter 
blocks could well have been used in the experimental simulation, which 
would have reduced the required storage capacity or increased the time 
to overflow. 


II. THE DATA CHANNEL 


A block diagram of the self-contained data channel is shown in Fig. 1. 
The transmitter consists of a buffer store, an encoder, a modulator, a 
reverse channel receiver and some logic. The transmission channel itself 
has a forward path and a reverse path, the latter carrying very little in- 
formation compared to the former. The receiver consists of a detector, 
a decoder, a buffer store, and a reverse channel transmitter plus logic. 

The forward channel carries data (plus any necessary redundancy and 
starting codes); the reverse channel carries information indicating 
whether retransmission is required. Errors in the reverse channel will 
not appreciably affect the operation. The small amount of information 
required over this channel permits a high degree of redundancy. In ad- 
dition, a “fail-safe” code can be used, so that any undetected errors on 
the reverse channel result in unnecessary retransmissions (subsequently 
eliminated at the receiver) to ensure against loss of data. 

To facilitate discussion, a specific model, chosen for its relative sim- 
plicity, is described. Modifications and improvements are apparent and 
will be briefly discussed. The method of operation is to accept data from 
the source continuously at a constant rate, Rs bits per second, which is 
less than the maximum rate, R, , allowed by the data transmission sys- 
tem. The efficiency then, without considering the error-detecting code, 
is 

E = Rs/Ri. (1) 


The data are transmitted at an effective rate of Rs until a retransmis- 
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sion is requested. After a retransmission request, data are sent at the 
higher rate, Rx , until the system is returned to normal. 

The change in rate could be made by switching the transmitting speed 
of the data set. Another method to achieve the data rate change is con- 
tinuous transmission at rate R, with interspersed dummy or “‘fill-in’’ 
bits as needed. The two methods are mathematically equivalent, and 
we shall assume the latter for the discussion in this paper. Thus, in the 
transmitting buffer the data are organized into blocks of N bits each 
and sent to the encoder at a rate, R, , faster than the maximum allow- 
able input rate. In order to equalize the input and output rates of the 
buffer, “‘fill-in’” bits containing no information are inserted between the 
blocks of message bits as shown in Fig. 2. The data then pass through 
the encoder, where additional redundancy is added to allow for error 
detection. At the encoder, one may either ignore the fill-in bits or en- 
code them, but will probably use them to transmit additional useful 
information. It is of course possible to place the error control encoder 
before the buffer, but this increases the required buffer size without gain- 
ing any apparent advantages. The signal is then modulated for transmis- 
sion over the forward path. 

Each block of information is retained in the transmitting buffer until 
it is certain that there will be no retransmission request from the re- 
ceiver. When a sufficient time interval has elapsed and no retransmis- 
sion request is received, the block of data is erased from the transmitting 
buffer. This time interval is taken to be 7p , the maximum round-trip 
delay for which the system is designed. This includes the transmission 
time in both directions plus any additional time for logical operations at 
either end. 

The system as described has a sort of natural block length, the number 
of bits emitted by the source at rate Rs in time T'p 


N = RsT'p ‘ (2a) 


With this block size, it is known that a retransmission request must apply 
to the immediately preceding block of data bits. 

It is shown later that shorter blocks have an advantage in reducing 
the required buffer size, and hence we let 


N = RsTp/k (2b) 


where & is an integer. For these shorter blocks, the system must assume 
a maximum 7’p or must include some provision for determining the ac- 
tual round-trip delay time so that retransmission requests can be associ- 
ated with the proper blocks of data. 
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Fig. 1 — Complete self-contained error control channel. 
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SOURCE MESSAGE (RATE Re) 
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k--Tp- > 
pits EFFICIENCY, E=Re6/R. 


BLOCK SIZE, N=RsTp 
TRANSMITTED MESSAGE (RATE R_) 


RETRANSMISSION 
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D 
| BITS | |-___»Tp | | 
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Fig. 2 — Example of time sequence at transmitter. 


The number of bits, including both data and fill-in, from the buffer 
in the same time 7')/k is 


N + M = RT p/k. (3) 


In the receiver the demodulated signal is decoded and checked for 
errors. If no errors are found, the data block, with all redundancy re- 
moved, is placed in the receiving buffer. In case an error is detected in 
the received block of data, a retransmission request is sent to the trans- 
mitter via the reverse data channel, and no data are sent to the receiv- 
ing buffer. 

In the transmitter we impose the operating rule: in case a retransmis- 
sion request is observed, the transmitter will complete the transmission 
of the current block of N + M bits and then revert to the beginning of 
the block detected to be in error.+ The transmitter then enters the re- 
transmission mode and retransmits information starting with the block 
in error. During this period, the transmitting buffer continues to receive 
and store data from the source, thus increasing the quantity of informa- 
tion stored. In order to return the transmitting buffer to its normal state, 
the fill-in bits are now omitted between the transmitted blocks of data, 
so that bits will be removed faster than they arrive. This reduces the 


+ Another way to say this is that the transmitter takes no action on a retrans- 
mission request until the end of a full round-trip delay time, 7'p , after sending the 
last bit of the block to be retransmitted. In this form the statement is also true 
when the transmitter is already in the retransmission mode. Note that in the latter 
case the time of decision is not necessarily at the end of a block. 
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information stored in the transmitting buffer and at the same time tends 
to refill the receiving buffer. The fill-in bits are omitted until both buffers 
have returned to their normal state. 

The above sequence is illustrated in Fig. 2. Block A has been received 
in error. The retransmission request is noted by the transmitter before 
the completion of block B. At the conclusion of block B transmission, 
both A and B are retransmitted. Fill-in bits are now omitted until such 
time as the transmitting buffer returns to its normal state. This occurs 
after transmission of block F, if there are no additional retransmission 
requests. 

We note immediately that, in case a number of nearby data blocks are 
found to be in error, the transmitting buffer may overflow. Similarly, the 
receiving buffer may empty out, so that for some time no information will 
be available to the data sink. The frequency of occurrence of these events 
depends, of course, on the error statistics of the channel, the storage 
capacity of the buffers, the round-trip transmission delay, the number of 
fill-in bits allowed between data blocks, and the size of the data block. 

Questions to be answered about the self-contained data channel are: 
How often does the transmitting buffer store overflow and the receiving 
buffer empty completely? What delay is encountered by the information 
prior to delivery to the sink? What efficiency can this system achieve? 
What buffer store capacity is needed? In general, what are the relation- 
ships between buffer store size, block length, transmission efficiency, 
transmission delay, and average time between overflows, in any given 
message? 


Ill. THE MARKOV PROCESS 


In the following development, it will be assumed that retransmission 
requests are independent with probability P,. For digital data trans- 
mission over telephone lines, individual bit errors are known to be not 
independent; however, for blocks which are long with respect to the bit 
error dependence, the retransmission requests will be nearly independent. 
There is some evidence that over voice telephone circuits at 1000-2000 
bits per second the correlation among bit errors becomes so small after 
10-15 bit intervals that the assumption of block error independence is 
acceptable.’ An estimate of the probability of a retransmission is avail- 
able, since the block error rate cannot be greater than the bit error rate 
times the block length. 


+ Let d be the bit error rate in B bits. Then \B is the number of bits in error. 
The number of blocks in error cannot be greater than \B. The total number of 
blocks is B/N so an upper limit of probability of block error is 
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We shall now devote ourselves to the question of the relationship be- 
tween the storage capacity of the buffer and the average time between 
overflow of the buffer. It is evident that, since the number of data bits 
transmitted per unit time is not constant, an actual time calculation is 
inconvenient. We therefore quantize time into unequal units, such that 
the number of data bits transmitted per quantum is always the same. 

The possible number of bits stored in the buffer form the states of a 
stochastic process. It will now be shown that, if these are considered only 
at certain moments of decision, the buffer states, y, form a finite Markov 
chain. 

The only time a decision is made is exactly 7'p seconds after the last 
bit of a block has been transmitted, and the decision consists of three 
parts: 

(a) Which block shall be transmitted? 

(b) Shall fill-in bits be transmitted following the data block? 

(c) May the transmitting buffer erase a block of data? 

The decision depends only on the state of the buffer and on whether a 
retransmission is requested; there are four cases: 

(t) Normal — The system ts not in the retransmission mode, and retrans- 
mission ts not required. The buffer erases one block; the transmitter sends 
fill-in bits and then the next block in sequence from the source. By the 
time of the next decision, the buffer will have replaced the erased block 
with one block from the source. Thus, at the moment of the next deci- 
sion, the total change in the buffer storage is zero. The time to the next 
decision is T'p/k. 

(it) The system is not tn the retransmission mode, but a retransmission 1s 
requested. The buffer does not drop any bits. The transmitter backs up 
to the block at the beginning of the buffer in order to retransmit the block 
received in error. The transmitter shifts its mode and no fill-in bits are 
sent. The next decision will be made after one block has been completely 
transmitted plus Tp seconds, to allow time for another retransmission 
request to be received. During the retransmission time, HN bits come 


AB 
a SAN 


There may be multiple bit errors in a block, and some of the block errors may not 
be detected, so 


P, SON. (4a) 
For the special case where bit errors are independent 
P,=1-— (1—A)¥ = ON. (4b) 


for \ much smaller than 1. 
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from the source, and during 7p , RsT'p bits. The total increase in storage 
due to one retransmission is thus 


I = RsTp + EN = ReTp(1 + E/k). (5) 


The time to the next decision is 7'p(1 + E/k). 

(tit) Off-Normal — The system has previously entered the retransmisston 
mode and no additional retransmission is requested. The buffer can drop 
the block which was received correctly. The transmitter continues with 
the block following the one just sent, without fill-in bits. The next deci- 
sion will take place after the time required to transmit one block, in 
which time EN bits are added to the buffer. Since the buffer has dropped 
a full block, the amount of data in the buffer has decreased by 


D=N(1 — B). (6) 


The time to the next decision is TpH/k. 

(iv) The system ts in the retransmission mode and another retransmission 
is required. This is similar to case (iz), except that the transmitter shift is 
not required since it is already in the retransmission mode. The same 
number of bits will be discarded at the receiver, but, being already in the 
retransmission mode, none of these are fill-in bits, so the number of 
blocks to be retransmitted is greater by the ratio (NV + M)/N. The trans- 
mitter remains in the retransmission mode and fill-in bits will not be 
sent. The increase in storage is given by (5), and the time to the next 
decision is 7p(1 + E/k). 

Let C be the total storage capacity of the transmitting buffer. When 
the source rate is constant, the transmitter can send the block as it is 
received. In this case, the smallest useful capacity, Cmin , includes the 
one block to which the retransmission request applies, if received, plus 
the data which arrive from the source during the round-trip delay 
preceding the request 


Crain =WN a RsTp . (7) 


If the source rate may fluctuate and the start of transmission must be 
delayed, C must be larger. The worst case is that in which the source 
may intermittently stop so the transmitter must wait until the full block 
is received, in which case the minimum C is one block more. This addi- 
tional block of storage to compensate for an intermittent source should 
probably not be charged to the error control system. The ability to pro- 
vide this feature in a simple manner is, however, an advantage of the 
system. 

There is another meaning for Cnin . In the normal mode of operation 
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there must be just this many bits in storage at each time of decision. 
In setting up the Markov states below, we do not count this irreducible 
storage, but it is included in the final results for total storage capacity. 

We have defined the state of the buffer, y, as the number of bits 
stored at any instant of decision. With a capacity of C bits, the range 
of this variable is 


O<ySCH1. (8) 


The normal state is y = 0; overflow is y = C + 1. 

We can now write down the transition probabilities, p;;, of going 
from buffer state y; to state y; . Starting in the zero or normal state, the 
buffer stays in the normal state with probability 1 — P, and increases 
by I with probability P, 


oo=1—P,, por = P,. (9a) 


If the buffer is within D states of normal, at the next decision it will 
either return to normal or will increase by I 


Pyo = 1—P,, Pyyst = P, for OSy SD (9b) 


If the buffer is more than D states from normal and more than J states 
from overflow, it will decrease by D or increase by J, but can neither 
return to normal nor overflow 


Dyy-p = 1 — P,, Dyytr = P, for D<ysSC-I. (9c) 


If the buffer is within J states of overflow, the buffer will either decrease 
by D or go to overflow 


Dyy-p =1—P,, Py,co = P, for C-I<y<C+l. (9d) 


In order to calculate the time to overflow, we force the buffer to stay in 
the overflow condition once it enters this state; i.e., the overflow state 
is made “absorbing” 


Pou, cH = 1. (9e) 


For all other transitions p;; = 0. The transition matrix is 
LT = {pis}. (9f) 


In addition, we let the process start in the normal state with prob- 
ability 1. The buffer state, in response to the retransmission signal, de- 
pends only on the buffer state at the previous moment of decision. This 
is the fundamental property for a process to be a Markov chain. 

A schematic representation of the Markov chain described by equa- 
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tions (9) is given in Fig. 3. The over-all operation of the transmitter 
may be seen in Fig. 4, which shows the internal state diagram of a se- 
quential machine which might be used to implement the transmitter. 
The states of the sequential machine are the same as the states of the 
Markov process, except that several of the latter may map into a single 
one of the former. 
The arrow labels — A,B/C,D — are identified as follows. In all cases, 
a dash means the item is immaterial. 
Transmitter inputs: 
A — Has a retransmission request been received? 
0 — no 1 — yes 
B— What is the state of the buffer? 
0 — empty (except for Cin) 
I — partially filled 
1 — over-filled 
Transmitter outputs: 
C — May a block be dropped from storage? 
0 — no 1 — yes 
D — Which block shall be sent next? 
D,-1 was the block which was just sent. D, is the next block 
in sequence, and D,_m is the mth block before. 
F, and F, are fill-in bits. Note that if ’; = I, , two states 
may be combined. 
Fig. 4 also applies to the receiver, except for reinterpretation of the 
labels. 
Receiver inputs: 
A — Has an error been detected? 


0 — no 1 — yes 
B — State of receiving buffer 

0 — full 

I — intermediate 

1 — empty 


Receiver outputs: 
C — Shall this block be sent to output store? 
0 — no 1 — yes 
D — Shall a retransmission request be sent? 
These will all be 0 except the two labelled D,-m and Dr—m—1 , 
which will be 1. 
For certain relations among the quantities involved, the matrix can be 
partitioned into several closed sets’ of states, such that it is not possible 
to make the transition from a state in any one closed set to a state in 
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Fig. 3 — Markov state diagram. 


any other such set. The states which cannot be entered from the normal 
state by any path may be removed from the matrix, thus reducing its 
size. This can be done by dividing out the greatest common factor in 
D, I, N, and C. A large number of the cases of interest are still included 
when this ‘normalizing factor’ is made equal to D. 


IV. CALCULATIONS 


Following the method outlined in Kemeny and Snell,’ we let Q be the 
transition matrix of all the transient states, i.e., matrix 7', excluding the 
overflow state. Let J be the identity matrix. Then 


G= (J —Q)” (10) 


exists and is called the fundamental matrix of the Markov process, with 
the following interpretation. Each element n;; of G is the mean number 
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Fig. 4— Diagram of internal states for transmitter. 


of times the process is in state 7, given that it started in state 7. With 7 = 
0 for starting in the normal state, the row sum over 7 is the mean number 
of times the process is in any of the transient states, from which we can 
calculate the mean time to the first overflow. Thus, the average number 
of decisions before overflow is 


(n) = 24 M0}. (11) 


Higher moments, in particular the second, can be found by additional 
operations on the fundamental matrix.’ 

A computer program was written to do the matrix arithmetic, and a 
few representative cases were solved numerically. The program computes 
the average number of blocks transmitted before overflow and the 
variance about this mean. The standard deviation is usually large, 
nearly equal to the mean. Typical examples are: when mean number of 
blocks before overflow was 23, standard deviation was 19; when mean 
was 949, standard deviation was 943; and when mean was 4795, standard 
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deviation was 4792. Thus the mean is a poor estimate of the actual time 
to overflow for any specific message, but is meaningful when a large 
number of transmissions are considered. 

The calculations to this point have been in terms of the number of 
blocks, and we now convert back to actual time. Instead of a straight 
sum on 7%; , we multiply each term by the actual time taken. 

There are four terms corresponding to the four cases described under 
the Markov process. The average time for each of the four cases is 


(2) Noo(1 — P,)Tp/k 
(14) noP,(1 + E/k) Tp 


(iti) 7 noj(1 — P,) ET /k 


Cc 
(iv) [> mj — | P.(1 + E/k)T . 
j=1 
The average time to overflow is the sum of these four: 


1—P, , EP; 
k is k 








) + 3 noi(P. + E/h) 
— P,(1 + E/k). 


lave 
Fr OU NM P, id 
Pp : ( (12) 


V. RESULTS 


As expected, the average time before the buffer overflows will increase 
when the buffer capacity is increased, and when the following variables 
are decreased: the bit rate, the round-trip delay, the probability of re- 
transmission, the efficiency, and the block size. The number of variables 
can be reduced by measuring time in units of 7’p , the round-trip delay, 
and bits in units of R,7T'p , the number of bits from the buffer in time 
Ty . Since the block error rate depends on the length of the block, the 
probability of retransmission is modified by the block length. The varia- 
bles of the system, all of which are now dimensionless, become 


C* — C/R1iT > 
N* — N/R1T p 
E 


P* = P,R,T)/N 
i* = t/T 


I 
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A number of curves are plotted to show the expected time between 
overflows as a function of the probability of retransmission. For each 
curve, the size of the buffer, the block size, and the efficiency are held 
constant. When the expected time, t/7'p , is greater than about 100 (cor- 
responding to several seconds of transmission for reasonable values of 
Tp), the curves are nearly linear on log-log paper, and only this portion 
is plotted. 

To use the curves, it is assumed that the transmission line parameters, 
R, and Tp , are known. In order to facilitate interpretation of the curves, 
some reasonable specific values have been assigned to these parameters 
and the corresponding values of time, buffer size, and block length have 
been calculated. The assignments are as follows: Let Ry, be 2000 bits 
per second; this could be a 2400 bps data set with an 833 per cent effi- 
cient error-detecting code. Let Tp be 120 ms. Then RiTp = 240 bits, 
the total number of bits sent in one round-trip delay time. Some other 
parameters are given in Table I. 

Fig. 5 shows the time gained by increasing the capacity of the buffer 
store. For this set of curves the efficiency is 0.5 and the block length is 
0.5 R,Tp ; that is, the block is as long as the maximum round-trip delay. 
When the efficiency is increased to 0.75 and 0.9, with the same block 
length (0.5 R,Tp), the results are as shown in Figs. 6 and 7, respectively. 
The storage capacity required to provide a specified time to overflow at 
a given probability of retransmission increases markedly with efficiency. 
The same effect is shown in Fig. 8, where the capacity is held constant 
for several efficiencies. The source bit rate at H = 0.75 is 50 per cent 
greater than at # = 0.5, and at H = 0.9 the bit rate is up by 80 per cent. 
The cost of this increased bit rate is either the extra buffer storage or the 
reduced time between overflows. Some of the data from Figs. 5-8 are 


TABLE J — OPERATING PARAMETERS 
(Given that Rr = 2000 b/s and 7'p = 120 msec) 





E : Rs N I Cin 
(bits/sec) (bits) (bits) (bits) 

0.5 1000 20 130 140 
120 180 240 

0.75 1500 20 195 200 
120 270 300 

180 315 360 

0.9 1800 20 234 236 
120 324 336 


216 410 432 
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Fig. 5 — Effect of buffer size: H = 0.5. 


shown in Table II, using the arbitrary assignments R;, = 2000 b/s, 
Tp = 120 ms, and N = 120 bits. 

In all the above cases, the block lengths have been the same, 0.5 RiT'p 
(120 bits). Only when the efficiency is 0.5 does this represent the so-called 
“natural” block, i.e., the number of bits from the source in one round- 
trip delay time; at the increased bit rates of the higher efficiencies, the 
natural block length is also increased. The effect of increasing the block 
length in one case is shown in Fig. 9, which can be compared to Fig. 6. 
The required capacity for a given time to overflow has increased mark- 
edly. We therefore investigate the effects of shorter blocks. 

Fig. 10 illustrates the case where each natural block is divided into 
three shorter blocks. A decision is made at the end of each arrow, and 
the fourth block back is either dropped from the buffer or is retrans- 
mitted. For example, when a retransmission is received while sending 
B;, both A: and Az have been dropped and 4A; is the next block to be 
sent. With sufficiently inexpensive logic in the terminals, improved per- 
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Fig. 6 — Effect of buffer size: H = 0.75. 


formance is possible on short loops by using the actual value of Tp . In 
the example, we might have already dropped A; and therefore start the 
retransmission with B,. 

In Fig. 11 we show the effect of decreasing the block size, at constant 
capacity and efficiency. Similar results for a larger capacity and efficien- 
cies of 0.5 and 0.75 are shown in Fig. 12. 

It is somewhat difficult to visualize all of these effects when plotted 
separately. We attempt to summarize some of the results in Fig. 13. 
For these curves the normalized retransmission probability, P*, is held 
constant, and buffer storage capacity is held to the minimum usable 
value, as given by (7); that is, the capacity is the natural block length 
plus the actual block length, and therefore decreases with either the block 
size or the efficiency. Both the latter are allowed to vary and we show 
the effect on the time to overflow. 

There is little effect from changing the block size — except on the 
buffer capacity. One would therefore choose the smallest practical block. 
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Fig. 7 — Effect of buffer size: HE = 0.90. 


However, as the efficiency is increased, the required buffer capacity is 
increased, although not rapidly, and the time between overflows de- 
creases. As shown earlier (Figs. 5-7, 9) it is possible to regain this loss in 
time to overflow by modest increases in buffer capacity over the minimum 
used here. Since the increased efficiency increases the maximum source 
rate, this is certainly the direction to go, up to the point where the in- 
creased rate is worth less than the cost of the additional storage required. 


VI. DELAY 


For smooth flow to the sink the receiving buffer must have the same 
capacity as the transmitting buffer, and will normally be kept full. Thus 
the receiving buffer will introduce a delay in the message of 


r= C/Rs. (14) 


This is in addition to the delay of T'p/2 from the transmission line. 
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Fig. 8 — Effect of efficiency: buffer capacity fixed. 


TABLE IJ — MEAN TIME TO OVERFLOW 
(Given that Rr = 2000 b/s, Tp = 120 msec, N = 120 bits) 


Ave. Time to Overflow (Hours) 


E = Rg/Ry C (bits) 

P* = 0,01 * = 0,001 
0.5 C'min (240) 0.67 66.6 
0.75 Cnin (300) 0.12 11.2 
0.9 Cnin (336) 0.03 2.90 
0.5 360 44.4 >1 year 
0.75 360 0.15 14.9 
0.9 360 0.04 3.14 
0.5 480 245.3 >1 year 
0.75 480 0.42 44.1 
0.9 480 0.06 5.32 
0.5 600 >1 year >1 year 
0.75 600 6.29 >1 year 
0.9 600 0.15 17.93 
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Fig. 9 — Effect of buffer size: longer block, E = 0.75. 


SOURCE MESSAGE (RATE Rs) 
A Re he EB labs Bg Ae Fe Ce ieee 
and J EFFICIENCY, E=Ro/RL 
Briar eee BLOCK SIZE, N=Rg1p/k, FOR k=3 


RETRANSMISSION 
TRANSMITTED MESSAGE (RATE R_) EQUEST 


Fig. 10 — Example of time sequence with shorter blocks. 
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Fig. 11 — Effect of block size: fixed buffer capacity, H = 0.5. 


There are other choices for operating the receiving buffer which will 
decrease the delay at the expense of irregularity of flow to the sink, which 
may be tolerable in many cases. If there were no receiving buffer at all, 
the delay would be zero except when retransmissions were required. 
When retransmissions are required, however, there would be additional 
delay until the block is received correctly, up to a maximum given by 
(14). The flow to the sink would not be smooth; each block would be 
delivered at rate R, , followed by an interval when no data are being 
delivered. Various compromises between these extremes are possible. 
For example, buffer capacity of a single block would permit data to be 
delivered to the sink at the source rate with no interruptions until a 
retransmission is requested. Then the sink must alternately wait and 
accept data at the higher line rate until the process returns to normal. 
The delay is variable, the minimum being 


with the maximum again given by (14). 


ERROR CONTROL 395 


TIME TO OVERFLOW IN HOURS 
100 

PARAMETER VALUES GIVEN: 
R, = 2000 BITS/SEC 
Tp = 120 MS 









































C=1.5 RLTp (360 Bits) 
E=0.5, 0.75 
N—VARIES 


PROBABILITY OF RETRANSMISSION, P.-Ri Tp/N 





106 2 5 


TIME, t/Tp 
Fig. 12 — Effect of block size: fixed buffer capacity, ZH = 0.5 and 0.75. 


This is one case where it has been possible to develop a calculable rela- 
tionship between the message delay involved in error control and the 
resulting error rate. 


VII. OTHER MODIFICATIONS 


The system may be designed to take any of several actions when an 
overflow of the buffer occurs. The source and sink may be stopped, re- 
quiring manual resetting; they may be temporarily halted for a time 
sufficient for the system to clear; or, without stopping the source, the 
uncorrected data block may be delivered to the data sink, with or with- 
out an indication that the particular block contains errors. 

One desirable modification would be to act sooner on receipt of the 
retransmission request. The transmitter would not continue to the end 
of the current block, but would immediately back up to the beginning 
of the block in error. This procedure could be quantized by using blocks a 
fraction of N in length. As indicated above, this procedure would require 
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Fig. 13 — Capacity and time to overflow as functions of efficiency and block 
length. 


either a knowledge of the actual round-trip delay or inclusion, in the re- 
transmission requests and the retransmission, of an indication of the 
exact block (or fraction) involved. Another modification which would 
improve performance on shorter loops would be to make a preliminary 
measurement of the round-trip delay and adjust the operation accord- 
ingly. This could be done automatically. 

Earlier, we mentioned the problem of an irregular input sequence and 
indicated that one additional block of storage was necessary. If this block 
is not counted, the performance level will be as given for a regular source, 
except for the possible gain arising from the probability of the intermit- 
tent source being stopped during the time when retransmissions are re- 
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quired. The output will be delayed an additional time corresponding to 
one block of data, but will be smoothed considerably — the rate will be 
constant except when waiting for the source. 

It has been assumed that, once an error is detected, all subsequent 
received data are ignored until that block has been retransmitted and re- 
ceived correctly. With more complicated bookkeeping it would be possi- 
ble to save some of these blocks, reducing the amount of retransmission 
required. On the other hand, since errors do occur in bursts on many 
transmission channels, the immediately succeeding block would have a 
higher-than-average error rate, and so might not be worth saving. 


VIIT. CONCLUSIONS 


It has been shown that it is possible to calculate the performance of a 
self-contained error-control system by treating the system as a Markov 
process when the system consists of (a) an error-detection code, (b) pro- 
vision for requesting and accomplishing retransmissions as necessary, 
and (c) buffer storage to allow smooth, uninterrupted flow from the 
source to the sink. Failure occurs when a sufficient number of retrans- 
missions are requested in a short enough time that the total information 
to be stored exceeds the capacity of the buffer. 

Whenever an overflow is about to occur, we could ignore the retrans- 
mission request and deliver the block as-is, in which case it appears to the 
sink as an error. It seems reasonable to require that this type of error 
should have about the same frequency of occurrence as undetected er- 
rors. For voice channels using reasonably simple codes, we might assume 
an undetected error rate of 10-8 or about one error per day.!? We might 
also require the efficiency to be about that of the error detecting code. 

With these criteria, it appears clear that one should not try to work 
with minimal storage, because of the relatively short time to overflow. 
Neither should one try to push the efficiency very high, or the required 
capacity grows out of bounds. A reasonable compromise for voice chan- 
nels would be a buffer capacity somewhat less than 1000 bits. 

We get a slightly different answer if we consider instrumentation. It is 
likely to be economically infeasible to build a buffer of this size with in- 
dividual bit storage devices, especially since serial access is adequate. 
However, with bulk storage such as a circulating delay line or a magnetic 
tape loop, moderate increase in buffer size is not costly, and several 
thousand bits would be available about as cheaply as a few hundred. 
This would permit buffer efficiencies close to unity. 

Results for any other specific cases can be easily calculated with this 
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computer program. It is apparent that a number of modifications in the 
model are possible and would serve to reduce the required storage. The 
transition matrix would merely have to be changed to correspond to the 
new model; the matrix arithmetic would be the same. 

The details of the chosen model and the examples were taken from a 
specific data transmission problem. The techniques, both the model and 
the method of solution, are applicable to a wider variety of problems 
where buffering is a consideration. 

We should like to acknowledge the assistance of H. O. Burton in con- 
sultation on the mathematics of the Markov process. We appreciate the 
continued encouragement of G. W. Gilman, who suggested the use of 
feedback error control with a data source which cannot be interrupted. 
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Intermodulation Distortion in Analog 
FM Troposcatter Systems 


By E. D. SUNDE 
(Manuscript received May 20, 1963) 


In broadband transmission over troposcatter paths, selective fading will 
be encountered with resultant transmission wmpairments, depending on the 
modulation method. An analysis has been made in a companion paper of 
such selective fading, based on an idealized model of troposcatter paths. It 
indicated that selective fading will be accompanied by phase nonlinearity 
which in a first approximation can be regarded as quadratic over a narrow 
band. A probability distribution for such quadratic phase distortion was 
derived. On the premise of quadratic phase distortion, the error probability 
owing to selective fading was determined for digital transmission by vari- 
ous methods of carrier modulation. 

The same idealized model and basic premise of quadratic phase distor- 
tton is used here to determine intermodulation distortion in FM for a sig- 
nal with the statistical properties of random noise. An approximate ex- 
pression for intermodulation noise owing to specified quadratic phase 
distortion has been derived, applying for any method of frequency pre- 
emphasis in FM. In turn, median intermodulation noise as well as the 
probability distribution of intermodulation noise has been determined, as 
related to certain basic system parameters. 

A comparison is made of predicted with measured intermodulation noise 
in four troposcatter systems with lengths from 185 to 440 miles. The results 
indicate that phase nonlinearity owing to selective fading can be approxi- 
mated by quadratic phase distortion, or linear delay distortion, over an 
appreciable part of the transmission band ordinarily considered for tropo- 
scatter systems, with a probability distribution that can be determined from 
certain basic parameters of troposcatter links, such as the length and an- 
tenna beam angles. However, to predict intermodulation distortion on any 
system, further experimental data than are now available are required on 
beam broadening by scatter. 

The present random multipath FM distortion theory 1s shown to afford 
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a significant improvement over an equivalent single-echo theory that has 
been applied on an empirical basis to troposcatter systems. 


INTRODUCTION 


An analysis has been made elsewhere! of error probabilities in high- 
speed digital transmission over idealized troposcatter paths, consider- 
ing both random noise and intersymbol interference owing to pulse 
distortion caused by selective fading. The above analysis indicated that 
a principal cause of intersymbol interference is a quadratic component 
of phase distortion, or linear delay distortion. On the same basic prem- 
ise an evaluation is made herein of intermodulation noise in analog 
transmission by frequency modulation, as now used for transmission of 
voice channels in frequency division multiplex. Expressions and curves 
are given of intermodulation noise in an idealized troposcatter channel 
for a signal with the properties of random noise, as related to certain 
basic system parameters and comparisons are made with the results of 
measurements on four troposcatter systems.?* 

In random multipath transmission the received wave can be considered 
the sum of a plurality of echoes, arriving over the various paths with 
varying amplitudes and different delays. Although this view is con- 
ceptually simple, it does not facilitate analysis of the statistical proper- 
ties of the received signal and of signal distortion. In the combination 
of a number of time functions, such as echoes, the analysis is greatly 
facilitated by the use of Fourier transformation to determine the cor- 
responding spectra. The latter can in turn be combined directly with 
appropriate attention to phase relations to obtain the resultant wave. 
For this reason it is preferable from the standpoint of analysis to regard 
the received wave as a multiplicity of sine wave components, rather 
than signal wave echoes, arriving over the plurality of transmission 
paths with varying amplitudes and phases. This is the method ordi- 
narily used in the analysis of the statistical properties of narrow-band 
random noise, which has properties that with appropriate translation 
of the basic parameters are also applicable to random multipath trans- 
mission. It is the method underlying both the previous determination 
of error probabilities in digital transmission owing to noise and selec- 
tive fading, and the present analysis of intermodulation noise in FM. 

In certain radio systems the received wave can be considered the sum 
of a principal signal wave and a weaker echo, and comprehensive theo- 
retical analyses have been published of intermodulation noise in FM 
owing to such echo distortion,’ »*:* together with the results of simulative 
tests.’ For these reasons this two-path model has been adopted as a 
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coarse simile to multipath transmission in some interpretations of the 
result of measurements of intermodulation noise in troposcatter sys- 
tems.? The limitations of this simile are recognized in the latter publica- 
tion,’ in which it is suggested that a more refined analysis is desirable. 
The idealized multipath model used in the analysis of troposcatter 
digital transmission affords a significant improvement, though it has 
certain predictable limitations, as shown herein. 


I. TRANSMITTANCE PROPERTIES OF TROPOSCATTER LINKS 


In tropospheric transmission beyond the horizon the received wave 
can be considered the sum of a large number of components of varying 
amplitudes resulting from a multiplicity of reflections within the com- 
mon volume of the antennas. Owing to variations in the structure of 
the common volume, caused largely by winds, there will be relatively 
slow changes in the many reflections and thus in the amplitudes of the 
component waves. When a steady-state sine wave is transmitted, the 
received wave will thus exhibit random variations in its envelope and 
phase, known as fading. 

In addition to such transmittance variations with time at a particu- 
lar frequency, there will be transmittance variations with frequency at 
any given instant, as illustrated in Fig. 1. At a given instant the ampli- 
tude and phase characteristics of the transmission path may be as indi- 
cated in Fig. 1(a) and at a later instant as in Fig. 1(b). 

Let u = w — wo represent the radian frequency relative to a reference 
frequency w). When the transmission vs frequency characteristic of a 
troposcatter channel varies slowly with time 1, it can be represented by 


(b) 


ATTENUATION ATTENUATION 





FREQUENCY, @—> FREQUENCY, @) ———> 


Fig. 1 — Illustrative variations in attenuation and phase characteristics with 
frequency at two instants ¢; and é2. 
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T(u,t) = A(u,t)e °%? (1) 
where 


A(u,t) = amplitude characteristic as a function of t for a fixed 
u, or as a function of u for a fixed time ¢ 
g(u,t) = phase characteristic. 


If u = wo is fixed, both A(uo, t) and y(t, ¢) are random variables of 
the time t, as are the time derivatives A’(wo,t), A” (wo, t), o’ (wo, ¢), 
vy” (uo , t). The probability distributions of A(w,¢) and ¢(uo, t) can be 
determined on the premise that they are the sum of a large number of 
randomly phased components. This results in a Rayleigh probability 
distribution of A(wo , ¢), in conformance with observations of rapid fad- 
ing. To determine the probability distributions of A’, A”, ¢’ and ¢”, 
statistical information is required regarding the rapidity of fades. This 
ordinarily takes the form of the time autocorrelation functions of A(t), 
or the related power spectrum of changes in transmittance amplitude. 
Such power spectra can be characterized by a certain equivalent fading 
bandwidth. 

If the time is assumed fixed at ¢ = ft, then A(wu,to) and ¢(u,to) will 
have certain random fluctuations with the frequency u that can be char- 
acterized by probability distributions. This also applies to A(u,to), 
A(u,to), o(u,to), and ¢(u,to), where the dots indicate differentiation with 
respect to frequency u. The probability distributions of A, ¢, and A and 
¢ depend on the frequency autocorrelation functions, or the correspond- 
ing power spectra of variations with frequency. The latter depend on 
differences in transmission time over the various paths, and can be re- 
lated to the maximum departure A from the mean transmission delay. 

The amplitude and phase characteristics as a function of u at any time 
fo can in general be represented by a power series as 


A(u,to) = ao + awe + au? + agu® + --- (2) 
o(u,to) = bo + bru + bow? + dbgu? + +>, (3) 


Certain basic relations have been developed by Carson and Fry’ and 
by van der Pohl,’ for transmission impairments in FM resulting from — 
attenuation and phase distortion. With the aid of these relations it can 
be shown that intermodulation noise is caused principally by phase dis- 
tortion rather than by amplitude distortion. Moreover, it can be shown 
that the principal contributor is quadratic phase distortion represented 
by bou’, which corresponds to linear delay distortion 2b.u. 
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II. PROBABILITY DISTRIBUTION OF QUADRATIC PHASE DISTORTION 
From (8) it follows that 


For u = 0, i.e., at the reference or carrier frequency, the probability 
distribution of bz is the same as that of ¢(0,t). The latter probability dis- 
tribution has been determined elsewhere’ on the approximate premise of 
a linear variation in transmission delay, with maximum departures +A 
from the mean delay. In Fig. 2 is shown the probability that ¢, or 2b: , 
exceeds A’/3 by a factor k. For example, there is a probability p = 0.5 
that ¢ exceeds A’/3 by a factor k & 1.2, and a probability p = 0.1 that 
g exceeds A’/3 by a factor k & 19. 

Thus in general 


op = 2b(p) = k,A’/3 (5) 


where k,A’/3 is the value of ¢, or 2b. with a probability p of being ex- 
ceeded. 
Alternatively, the value of bz with a probability p of being exceeded is 


b(p) = ae (6) 
Thus 
bo(0.5) = A? = 022° (7) 
be(O.1) a = 3.2A° (8) 
b2(0.01) ae A? = 67A2 (9) 


Thus, when A is known, together with intermodulation noise for 
quadratic phase distortion, it is possible to determine the median value 
of average intermodulation noise, or the value exceeded with any other 
specified probability p. 


III. INTERMODULATION NOISE FROM QUADRATIC PHASE DISTORTION 


In a first-order evaluation of intermodulation noise, only the quadratic 
term bou’ in (3) would be considered, since it will be the principal con- 
tributor. The ratio of nonlinear distortion power to average signal power 
at the frequency w will depend on the signal properties and on the pre- 
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Fig. 2 — Probability that ¢ or 2b2 exceeds A?/3 by a factor k. 


emphasis used in frequency modulation. It will be assumed that the orig- 
inal message wave has a flat power spectrum of radian bandwidth 
Q = 27rB and the statistical properties of random noise, and furthermore 
that the message wave is passed through a transmitting filter with a 
power transfer characteristic 


i(w) = 1 + c(w/Q)’ 
; (10) 
= 1+ c(f/B)’. 
At the receiving end a complementary filter is used to restore the mes- 


sage wave. 

As discussed in the Appendix, exact determination of intermodulation 
noise from quadratic phase distortion presents formidable difficulties, 
except on the premise of slight phase distortion, which is not generally 
applicable to troposcatter systems. However, it is possible to obtain an 
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approximate solution without the above limitation. The following rela- 
tion is derived in the Appendix for the ratio p(f) of intermodulation 
noise to average signal power at the frequency f = w/2r 


olf) =F @(c,a)H(y) (11) 


where c is defined by (10) 


a =f/B = «0/2 
B = bandwidth of baseband signal = Q/27 
D = rms frequency deviation = Q/27 
and 
y = dQ? = (2Q7)*bD”. (12) 


The function G(c,a) depends on the pre-emphasis and is given by 
expression (108) in the Appendix, which is 


3a 
G(c,a) = (i + ca®)(3 +c) F(e,a) 
F(ca) =2—-a+ 2e+ ea yy + (1 — a)’ (13) 


3 
ca 4 c 5 
-Sh-GQ-s)l+e0+a-—a)4 


This function is shown in Fig. 3 for pure FM and PM and for c = 16. 
The particular case of c = 16 and a = 1 will be considered further in 
the following, and for this case 


G(16,1) = 0.192. 


The function H(y) is shown in Fig. 4 and represents an approximation, 
as discussed in the Appendix. It will be noted that this function departs 
from proportionality with y’ for y = 0.5, reaches a certain maximum 
value and then diminishes. 


IV. INTERMODULATION NOISE IN TROPOSCATTER PATHS 


In accordance with (6), the value of be with a probability p of being 
exceeded is bo(p) = kpA’/6. The corresponding value of y is given by 
(12) as 

kd” 272 
y= —— (Qa) D 
lad (14) 
= 6.6k,(AD)’. 
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Fig. 3 — Function G(c,a) for pure FM (c = 0), pure PM (c = ), and for pre- 
emphasized FM with c = 16. 


Thus 
| 0.6 & 8 (AD)? (15) 
vo.1 & 125 (AD)? (16) 
v0.01 = 2600 (AD’). (17) 


The corresponding ratios p(f) at f = B with a probability p of being 
exceeded 


pp(B) = 0.192 (7) Hy) (18) 


oos(B) = 0.192 (7) H(8A°D*) (19) 
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po.1(B) = 0.192 (2) mcazsa%o' (20) 
po.n(B) = 0.192 (2) 120000"). (21) 


V. DIFFERENTIAL TRANSMISSION DELAY A 


Exact determination of the equivalent maximum departure from the 
mean transmission delay requires consideration of the antenna beam 
patterns as affected by scattering. On the approximate basis of equiva- 
lent antenna beam angles a, it follows from the geometry indicated in 


Fig. 5 that 
le OB a+ B 
ARS 5 (0+ 5 ) (22) 








where 8 S a, v is the velocity of propagation in free space, L is the length 
of the link, and 


2R = - 2RoK 8) 
where Rp is the radius of the earth and the factor K is ordinarily taken 
as 4/3. 

The equivalent antenna beam angle a from midbeam to the 3-db loss 
point depends on the free-space beam angle ap and on the effect of scat- 
ter, which is related in a complex manner to ap and the length JZ, or al- 
ternatively 6. Narrow-beam antennas as now used in actual systems are 
loosely defined by ao S 26/8. For these, a & ao on shorter links, while on 
longer links a > ap owing to beam-broadening by scatter. Analytical 
determination of a for longer links appears difficult, and only limited ex- 
perimental data are available at present. For broad-beam antennas, 
ao >> 26/3 and beam-broadening by scatter is in theory inappreciable. 

By way of numerical example, let L = 170 miles and K = 4/3, in 
which case @ = 0.016 radian. With ap = 0.004 radian « 26/3 it is per- 
missible to take a = ao. With 8 = a = ao, (22) gives A = 0.08 X 10° 
second. 

The differential delay A in general varies with time and for narrow- 
beam antennas can be considered the sum of two components 


A(t) = Ao + Ax(é) (24) 


where Ao is a fixed component obtained from (22) by taking a= a, 
the free-space beam angles. The variable component Aj(¢) depends on 
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Fig. 4 — Function H(y). The parameter y is the phase distortion in radians 
at a frequency corresponding to the rms frequency deviation @ = 27D radians/ 
second. 
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Fig. 5 — Definition of antenna beam angles a, take-off angle 8 and chord angle 
6 to midbeam. With different angles at the two ends, the mean angles are used in 
expressions for A. 
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scatter variation with time, as does path loss, and will have a certain cor- 
relation with path loss variations. Owing to the fixed component Ao, a 
weaker correlation exists between A(t) and path loss variations. 
Because of the dependence of A on path loss, the ratio p, of intermodu 
lation noise to average signal power will depend somewhat on path loss 
However, for a given path loss p, is independent of the average trans- 
mitter power and thus of the average signal power at the receiver. 


VI. LIMITATIONS ON FIRST-ORDER DISTORTION THEORY 


The above first-order approximation applies for sufficiently narrow 
signal bandwidths at the detector input such that terms in (3) of higher 
order than wu’ can be neglected. Results given by Rice for random vari- 
ables (Section 3.4 of Ref. 10) indicate there is no correlation between ¢ 
and %, so that distortion owing to the term b3u’ will combine on a power 
addition basis with distortion resulting from bow’. Moreover, there is a 
negative correlation factor between ¢ and‘¢, so that on the average b 
is negative whenever b, is positive, and conversely. Hence distortion pro- 
duced by byw‘ will on the average subtract directly on an amplitude basis 
from that resulting from bou’. In the range where the function H(y) 
increases linearly with y’, intermodulation noise owing to the term bw’ 
increases as b:’(AD)*. In the same range, intermodulation noise from the 
term b4u° will vary as bs(AD)8 and may hence have a significant effect 
for adequately large values of AD even though bs be much smaller than 
be . As shown later, comparisons of measured intermodulation noise with 
predictions based on the above first-order theory indicate the increasing 
importance of the term by" in reducing intermodulation noise as AD is 
increased. 


VII. TWO-PATH VS MULTIPATH DISTORTION THEORY 


The above first-order distortion theory is a mathematically derived 
approximation that in principle yields valid results with appropriate 
limitations on signal bandwidth and frequency deviation, and which 
retains the multipath feature that is essential to this end. By contrast, 
the two-path or single-echo simile mentioned in the introduction has 
no such basis but has been adopted principally because of the conven- 
ience of available theoretical analysis.*:*:® A second reason is that single- 
echo distortion theory yields results that in some respects are quite 
similar to those obtained with multipath transmission, as shown below. 

It is noteworthy that, by proper choice of echo amplitude and delay, 
results similar to those for median quadratic phase distortion can be 
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obtained. This is illustrated in Fig. 6, which shows the median ratio 
p(B) obtained from (19) as a function of D for B = 1 me/sec with 
A = 0.1 and 0.5 microsecond. In the same figures are shown the ratios 
p(B) obtained on the premise that the received wave consists of a main 
signal and an echo of equal amplitude delayed by 0.07 and 0.4 micro- 
second. The ratio p(B) for the latter condition is obtained from a chart 
given in Fig. 9 of Ref. 3, applying for FM with virtually the same pre- 
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INTERMODULATION NOISE IN DECIBELS BELOW AVERAGE SIGNAL POWER 


RMS DEVIATION, D, IN MEGACYCLES PER SECOND 


Fig. 6 — Comparison of intermodulation noise from single-echo distortion and 
quadratic phase distortion at B = 1 mc/sec: (solid lines) median intermodulation 
noise from quadratic phase distortion for indicated departures A from mean delay ; 
(dashed lines) intermodulation noise from echo of same amplitude as signal with 
delays A, as indicated. 
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emphasis as assumed herein and given by (10). The above charts are 
based on echo distortion theory applying for echoes that are much 
weaker than the signal, but this premise is ignored here in extending 
the theoretical results to a fictitious echo of the same amplitude as the 
signal. In this connection it may be noted that simulative tests® indicate 
that intermodulation noise is nearly proportional to echo amplitude, 
even when the latter equals the signal amplitude. With both quadratic 
phase distortion and single-echo distortion, intermodulation noise is 
virtually proportional to the second power of signal bandwidth. Hence, 
the relative comparisons in Fig. 6 could also apply for other bandwidths 
than B = 1 me/see. 

The above comparisons indicate that in applying equivalent single- 
echo FM distortion theory to multipath transmission as in troposcatter 
systems, with physically tenable echo delays, certain dilemmas will be 
encountered. The theory could be extended beyond its validity to fic- 
titious echoes of the same amplitude as the signal, to obtain virtually 
the same median intermodulation noise as for quadratic phase distor- 
tion. This would exclude the possibility of greater intermodulation noise 
than the median value, since the greater echo is by definition the main 
signal. The other procedure would be to assume an echo that is smaller 
than the main signal, which is physically more acceptable and does not 
violate the basic premise underlying echo distortion theory. In this case 
intermodulation noise predicted on the basis of echo distortion theory 
would, at least in certain cases, be much smaller than actually observed 
and could not be made to conform with observations, unless the echo 
amplitude is increased to the same amplitude as the signal. 

Thus, if the ratio of echo amplitude to signal amplitude is r, inter- 
modulation noise power based on single-echo theory will be less than 
for multipath transmission by a factor 7?. Hence it becomes necessary 
to introduce a factor 1/r? to make single-echo theory applicable to multi- 
path transmission. In Ref. 3, this factor has been determined empirically 
from measurements to be discussed later, and is given as 9 db. 


VIII. OBSERVED MEDIAN INTERMODULATION NOISE 


Measurements have been made on four troposcatter links of the me- 
dian value of intermodulation noise at the frequency f = B. The modu- 
lating wave in these tests had a flat power spectrum, and pre-emphasis 
was used that closely corresponded to c = 16 in (10). 

The basic parameters of the systems on which the measurements 
were made are given in Ref. 3 and are summarized in Table I. In this 
table ao is the free-space antenna beam angle from midbeam to the 3-db 
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TABLE I — Basic PARAMETERS OF TROPOSCATTER TEST 
SysTEMsS IN CARIBBEAN (A) AND IN Arctic (B,C,D) 


System A B , Cc D 
Length, miles 185 194 340 440 
Radio frequency, mc 725 900, 900 800 
Antenna/diameter, ft 60, 60 30,60 | 120, 120 120, 120 
ao (radian) 0.0115 0.017 |: 0.0058 0.0058 
6 (radian) - 0.015 . 0.016. | .. 0.0381 0.034 


we Ne el 


Ay (microsecond) 0.12 0.21") ° 0.185 | 0.255 


loss point, which may not conform with the angle a in (22) when scatter 
is considered. The values of K and 6 are taken from Ref. 3, and @ differs 
slightly from that obtained from (23) owing to differences in antenna 
elevations. The take-off angle @ is virtually zero and has been neglected. 
The value Ao of A given in the table was calculated with a = ao, rather 
than the actual beam angle with scatter. Systems A, B, C and D corre- 
spond to paths 1, 2, 4 and 3 in Ref. 3. 

In Figs. 7 and 8 are shown the ratios p; (B) expressed in db as a func- 
tion of the rms frequency deviation D for different bandwidths B of the 
baseband signals. 


IX. COMPARISON OF THEORETICAL WITH OBSERVED MEDIAN VALUES 


In the same Figs. 7 and 8 are shown median values of intermodula- 
tion noise obtained from (19) for each case, based on values A, of A 
that afford the best average approximation to the measurements. The 
latter values are somewhat greater than Ao, as indicated in Table IT. 

A ratio Am/Ao OF &m/ao > 1 is to be expected owing to beam-broaden- 
ing by scatter, and the above ratios appear reasonable in the light of 
present knowledge. Thus, if the actual angles a were known so that A 
could be determined, it appears plausible that satisfactory conformance 
with observed intermodulation noise would be obtaincd. 

As noted in Section V, A includes a component A,(¢) that varies with 
time depending on scatter conditions and which is correlated with path 
loss fluctuations. The ratio p thus depends on path loss as affected by 
scatter and has a certain correlation with path loss variation, as shown 
elsewhere.’ Hence, if measurements had been made under different 
path loss conditions, the derived values A,, would have been somewhat 
different. . 

From Figs. 7 and 8 it will be noted that with the above choice of A = 
A» it is possible to obtain better agreement between predicted and ob- 
served intermodulation noise for small bandwidths B of the baseband 


INTERMODULATION DISTORTION ; 413 


















































NOISE IN DECIBELS BELOW AVERAGE SIGNAL POWER 








60 
100 200 400 600 1000 2000 100 200 400 600 1000 2000 
RMS FREQUENCY DEVIATION IN KILOCYCLES PER SECOND 


Fig. 7 — Comparison of measured and calculated median intermodulation 
noise: (dashed curves) measured median intermodulation noise in top channels 
at indicated frequencies in ke; (solid curves) calculated median intermodulation 
noise for idealized model with the following values of the equivalent maximum 
deviation A from the mean transmission delay: system A, A,, = 0.12 microsecond 
(Ao = 0.12); system B, Am = 0.25 microsecond (Ay = 0.12). 


signal and small deviations D than for large bandwidths and frequency 
deviations. This probably resides in the circumstance that the phase 
distortion terms of higher order than byu? have been neglected in the 
above first-order theory, as discussed in Section VI. 

The measured median ratios given in Figs. 7 and 8 are plotted in 
Fig. 9 against the ratios predicted by first-order theory. It will be noted 
that measured intermodulation noise is less than predicted for signal- 
to-interference ratios less than about 30 db, owing to reduction in inter- 
modulation noise by phase distortion of higher order than bu? that has 
been neglected in first-order theory. The results in Fig. 9 permit an 
approximate empirical correction to first-order theory. 

As discussed in Section VII, with single-echo distortion theory vir- 
tually the same median intermodulation noise is obtained as with the 
above first-order theory, provided the echo is equal in amplitude to the 
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Fig. 8 — Comparison of measured and calculated median intermodulation 
noise: (dashed curves) measured median intermodulation noise in top channels 
at indicated frequencies in ke; (solid curves) calculated median intermodulation 
noise for idealized model with the following values of the equivalent maximum 
deviation A from the mean transmission delay: system C, Am = 0.25 microsecond 
(Ay = 0.185); system D, Amn = 0.55 microsecond (Ao = 0.255). 


mean signal. For smaller echoes, predicted intermodulation noise must - 
be less. This conforms with results presented in Figs. 12 and 14 of Ref. 
3, which show that intermodulation noise predicted from single-echo 
theory is significantly smaller than observed. To obtain a satisfactory 
average relation between predictions and observations, the predicted 
values must be increased by 9 db, as in Fig. 15 of Ref. 3 


TaBLE IT — Ratio An/Ao 


System A B Cc D 
Length, miles 185 194 340 440 
Ao, microsecond 0.12 0.21 0.185 0.255 
Am , microsecond 0.12 0.25 0.25 0.55 
Am/Ao 1.0 1.2 1.35 2.15 
am/ao 1.0 1.1 1.35 2.15 
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MEASURED MEDIAN SIGNAL-TO-INTERFERENCE RATIO IN DECIBELS 








MEDIAN SIGNAL-TO-INTERFERENCE RATIO WITH FIRST-ORDER THEORY IN DECIBELS 


Fig. 9 — Comparison of measured median signal-to-interference ratios with 
median values based on first-order approximation with best choice of differentia] 
transmission delay A. 


X. PROBABILITY DISTRIBUTION OF INTERMODULATION NOISE 


From (18) it is apparent that the probability distribution of p is di- 
rectly related to that of H(y,). This function is shown in Fig. 10 as re- 
lated to (AD)? for p = 0.5, 0.1 and 0.01. It should be recognized that 
this function as given herein is approximate, and that the errors are 
likely to be greater for small values of p than for median intermodula- 
tion noise as considered previously. ; 

From the curves in Fig. 10 it is possible to obtain approximate curves 
of the probability distribution of intermodulation noise, applying for 
various values of A?D? as shown in Fig. 11. These curves show that the 
probability distributions vary markedly with the above parameter, in 
conformance with a few probability distributions derived from observa- 
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Fig. 10 — Function H(y,) for various probabilities p. 


tions.? Because of the approximations involved in the present first-order 
distortion theory, the above probability distribution curves should be 
considered illustrative and may not be accurate enough for certain 
engineering applications. 


XI. PREDICTION OF INTERMODULATION DISTORTION 


The present first-order intermodulation theory indicates that inter- 
modulation distortion depends on the delay difference A, and this would 
apply also for an exact theory. For various troposcatter links with differ- 
ent angles a and 0, intermodulation distortion would be the same for 
equal values of A. This is exemplified by comparison of intermodulation 
noise in systems B and C as shown in Figs. 7 and 8. Though these sys- 
tems have different angles ao and 6, intermodulation noise is virtually 
the same since A is the same. Thus, if A could be determined, the above 
first-order theory, in conjunction with the above experimental data, 
would permit determination of intermodulation distortion for a variety 
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11 — Probability distributions of intermodulation noise for various values 
of (aby: corresponding to dashed lines a, b, c and d in Fig. 10. 


of conditions other than those in the tests. The above experimental 
data were confined to intermodulation noise in the top channel, i.e., 
fora = 1 in Fig. 3, and for a particular pre-emphasis, c = 16. The ex- 
pression for G(c,a), or the curves in Fig. 3, permit approximate deter- 
mination of intermodulation noise at other frequencies, and also for 
other kinds of pre-emphasis. For example, for a = 0.8, intermodulation 
noise would be greater than for a = 1 by an approximate factor 0.32/ 
0.19 ~ 1.7. If pure FM(c = 0) had been used in the tests, intermodula- 
tion noise at a = 1 would have been increased by an approximate fac- 
tor 1/0.19 & 5.2. 

At present there is a principal obstacle to prediction of intermodula- 
tion distortion for other values of a than in the above experimental 
systems. This is the lack of comprehensive experimental data on the 
beam angle a as affected by scatter for troposcatter links of various 
lengths. When and if such data become available, it will be possible to 
determine A and in turn intermodulation distortion in the manner indi- 
cated above for any kind of system. 
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XII, APPLICATION TO DIGITAL MULTIBAND TRANSMISSION 


The distributions in Fig. 10 apply for average intermodulation noise 
over brief time intervals, as determined by changes in phase distortion 
with time. During each such interval the instantaneous amplitudes of 
intermodulation noise will fluctuate about the average value. For a 
signal with the properties of random noise, as considered here, the proba- 
bility distribution of this fluctuation is approximated by the normal law. 
The distribution of instantaneous amplitudes or intermodulation noise 
is important in transmission by FM of a number of digital channels in 
frequency division multiplex, as discussed below. 

In digital transmission over troposcatter paths, the error probability 
for a given signal-to-noise ratio of the receiver depends on the trans- 
mission rate, as discussed elsewhere.!.As the transmission rate is in- 
creased, the error probability is ultimately determined by intersymbol 
interference owing to selective fading, and may be excessively high. 
The error probability can in this case be reduced, for a given total trans- 
mitter power, by transmitting at a slower rate over each of a number 
of narrower channels in frequency division multiplex. This could be 
accomplished by individual transmission over each channel, which 
would entail a number of independent transmitters. An alternative 
method would be to use a common amplifier and to transmit the com- 
bined digital signal by frequency modulation of a common carrier, as 
now used for transmission of voice frequency channels in frequency 
division multiplex. In the latter case, it is necessary to consider the 
possibility of additional transmission impairments owing to intermodu- 
lation noise. 

With a sufficiently large number of digital channels in frequency divi- 
sion multiplex, the combined wave will have virtually a Gaussian ampli- 
tude distribution, like random noise. Hence the probability distribution 
of average intermodulation noise amplitudes would be as indicated in 
Fig. 11 for various conditions. The instantaneous amplitude will fluctu- 
ate with respect to the above average values, as noted in Section X. 

In binary transmission it is often assumed that the error probability 
will not be excessive if the average noise power from all sources is about 
12 db below the average signal power, or 18 db below the peak signal 
power in on-off binary pulse transmission. From the previous curves 
and expressions it appears that intermodulation noise power averaged 
over short intervals will be at least 10 db below the average signal power, 
with a small probability that it exceeds —15 db. It thus appears that 
intermodulation noise will not be a limiting or predominant factor even 
when a large number of binary channels are combined in frequency 
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division multiplex for transmission by frequency modulation of a com- 
mon carrier. 


XIII. SUMMARY 


In broadband transmission over troposcatter paths, selective fading 
will be encountered with resultant transmission impairments, depending 
on the modulation method. A previous analysis has been made of such 
selective fading, based on an idealized model of a troposecatter path. It 
indicated that selective fading will be accompanied by phase distortion 
that in a first approximation can be regarded as quadratic, and a proba- 
bility distribution curve for such quadratic phase distortion was derived. 
On the premise of such quadratic phase distortion, the error probability 
owing to selective fading was determined for digital transmission by 
various methods of carrier modulation. 

In the present study the same basic premise of quadratic phase dis- 
tortion has been used in determining intermodulation distortion for a 
signal with the properties of random noise, based on the same idealiza- 
tion of a troposcatter path. An approximate relation for intermodulation 
noise owing to quadratic phase distortion has been derived, applying 
for any frequency pre-emphasis in FM. In turn, median intermodula- 
tion noise as well as the probability distribution of intermodulation 
noise has been determined, as related to certain basic system parame- 
ters. 

Median intermodulation noise predicted on basis of free-space antenna 
beam angles conforms well with observations on links 185 and 194 miles 
in length. For links 340 and 440 miles long it is necessary to use antenna 
beam angles that are greater than the free-space angles by factors of 
about 1.85 and 2.15, respectively. On long links employing narrow-beam 
antennas, beam broadening is expected because of scatter. Thus if the 
beam angles had been determined by independent observations or by 
more elaborate theory, it is probable that predicted intermodulation 
noise would conform reasonably well with observations. 

The results of intermodulation noise measurements thus appear to 
confirm the conclusion in a previous theoretical analysis of troposcatter 
transmittance, which indicated that phase distortion owing to selective 
fading could in a first approximation be represented by a component of 
quadratic phase distortion, with a probability distribution that can be 
determined from certain basic system parameters. This affords a simpli- 
fied first-order theoretical model of selective fading in troposcatter paths 
that is applicable to evaluation of resultant transmission impairments 
in both analog and digital transmission. 
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It can be shown analytically, and it is confirmed by observations, 
that the above first-order distortion theory yields intermodulation 
noise that in the case of large signal bandwidths and frequency devia- 
tions will be greater than observed or obtained with a more exact dis- 
tortion theory. An empirical curve presented here permits determination 
of the expected correction for large bandwidths and frequency devia- 
tions. a ee . 

It has also been demonstrated that the first-order multipath distor- 
tion theory presented here affords a significant improvement over single- 
echo distortion theory applied to random multipath transmission, in 
that it is simpler and accounts for the probability distribution of inter- 
modulation noise without certain contradictions that are inherent in 
single-echo theory. Taken in conjunction with presently available data 
on observed intermodulation noise on certain troposcatter links, as dis- 
cussed herein, it affords a means of predicting intermodulation noise on 
any system when more comprehensive experimental data become avail- 
able on antenna beam broadening by scatter. 


APPENDIX 


Intermodulation Noise from Quadratic Phase Distortion in Pre-Emphasized 
FM 


General 


To facilitate analysis of intermodulation noise in FM owing to attenu- 
ation and phase distortion, it is customary to introduce two basic ap- 
proximations. One is the use of ‘‘quasistationary theory” in conjunction 
with the concept of instantaneous frequency, which is permissible when 
the signal bandwidth B is negligible in comparison with the carrier fre- 
quency, so that the frequency changes imperceptibly over a signal in- 
terval T’ = 1/2B. The other customary approximation is that distortion 
a(w) + 7i8(w) is sufficiently small to permit the approximation exp 
[—a(w) — 78(w)| & 1 — a(w) — 78(w) over the bandwidth of the 
modulated carrier wave. The latter is a legitimate approximation for 
most transmission systems, and greatly simplifies the analysis, but may 
lead to appreciable errors in applications to tropospheric paths where 
pronounced attenuation and phase distortion can be encountered. For 
this reason an alternative approximate analysis is adopted herein to de- 
termine intermodulation noise from quadratic phase distortion, in which 
no limitation is placed on the phase distortion. 

Two limiting cases are considered, from which it is possible to make an 
approximate determination of intermodulation noise as related to phase 
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distortion, rms frequency deviation, and bandwidth of the baseband 
signal. In the first case, phase distortion is assumed adequately small, 
such that the maximum phase distortion in the carrier signal band is less 
than 7 radians. Under this condition it is possible by use of ‘‘quasista- 
tionary” theory to determine the power spectrum of intermodulation 
noise without much difficulty. In the second case, no limitation is placed 
on phase distortion, in which case determination of the power spectrum 
becomes excessively difficult or laborious. It is possible, however, to de- 
termine total intermodulation noise power at the detector output, prior 
to post-detection low-pass filtering. From the manner in which total in- 
termodulation noise power behaves with increasing phase distortion, it 
is possible to obtain an approximate evaluation of intermodulating noise 
in a narrow band, such as a voice channel. 


A.1 Power Spectrum of Phase Modulation 
In FM the transmitted wave is of the general form 
V = cos [wot + (t)] (25) 
where the phase y(t) is related to the modulating wave m(t) by 


t 
wit) = bf mit) at (26) 
: 0 
where & is a constant. 
The instantaneous frequency deviation is accordingly 
Ot) = Wt) = km(t). (27) 


If the original signal wave has a power spectrum s(w) and power pre- 
emphasis p(w) is used, the power spectrum of the modulating wave is 


Wnlw) = s(w)p(w). (28) 
The squared rms frequency deviation y’(¢) is 
=k | s(o)p(w) do. (29) 
0 


In accordance with (26), Y(t) is the integral of m(t). Hence the power 
spectrum of ¥(¢) is given by 


Wy(w) = k’s(w)p(w)/w. (30) 
From (29) and (30) 
8(w) p(w) /w? 


[ s(o)plw) do 


The power spectrum of y(t) is w Wy(w). 


Wy(w) = & (31) 
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A.2 Autocorrelation Function of Phase Modulation 


The autocorrelation function of y(¢) is 


RGSS I Wisreoat ae (32) 


=k iE ay cos wr dw. (33) 


When the constant k is determined from (29), the following relation 
is obtained 


Ror) = a8 [RO cos or do | /| [ s(e)p(e) do]. (34) 


When the baseband power spectrum s(w) has a bandwidth Q, (34) can 
be written 


Ry(r) = w Ga oo COS wr w|/{[ s(w) p(w) aa | (35) 


where yp is the rms deviation ratio 
“= 9/9 = D/B. (36) 


In the special case of a flat power spectrum, s(w) = s and (35) yields 
2 2 
Ry(r) = ic E [ plo) COS wr iw|/| | p(w) i. (37) 
-0 Q) 0 
With pure FM, p(w) = p = constant and (37) reduces to 


1 
R,(r) = “ef cos Qrx Ag (38) 


x? 





where x = w/Q. From (38) it follows that 


R,(0) — R,(r) = “f 1 — cos Ore 4. 








2 
= p [Or Si(Q7) + cos Q7 — 1] (39) 
— a sue E = se ae | 


where $1 is the sine integral function. 
With pure PM, p(w) = o’ and (37) yields 


1 
2 
= Q 
Ry(r) = 3u I cos Ora dx a 


= 3u’ sin Q7/Qr 
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Ry(0) — Ry(r) 3u7[1 —sin Qr/Q7] 


: (ary fi 7 (ar) i a]. (41) 


I 








A.3 Intermodulation from Phase Distortion 


It will be assumed that the phase characteristic is of the form 


o(u) = bo + bu + dou? + dui + ee. (42) 
Phase distortion is then represented by the term 
B(u) = bow? + dbyu® + --- (43) 


where u = w — wo is the frequency relative to the carrier frequency wy . 
When the transmitted wave is of the form (25), the instantaneous 
frequency deviation is 


u(t) = '(t) (4) 
and the corresponding variation in phase distortion with time is 
Blu(t)] = dele’ (t)P + daly’(t) +--+. (45) 


In the above relation ¥’(¢) is given by (27) and the power spectrum of 
y(t) by (28) multiplied by k’ or 


Wy(u) = k’s(w)p(w). (46) 


In determining intermodulation distortion it must be recognized that 
distortion increases in the range 0 < 6[u(t)] S 7, diminishes in the range 
a < Blu(t)] < 27, increases in the range 27 < B[u(t)] < 37, ete., as 
illustrated in Fig. 12. 

To determine intermodulation distortion it is thus necessary to evalu- 
ate the distortion obtained when a wave with the power spectrum (46) is 
applied to a device with the output vs input characteristic illustrated 
in Fig. 12. Two limiting cases will be considered below. 


AA Intermodulation Spectrum for Small Quadratic Phase Distortion 
With quadratic phase distortion only, (45) becomes 


Blu(t)] = bly’ (ty). (47) 


It will be assumed that the probability that B[u(t)] exceeds z is so small 
that it is permissible to assume B[u(t)] < 7, and furthermore that u(t) 
changes at a sufficiently slow rate such that @’[u(t)| = 2be’(t) << x. 
For signals with the properties of random noise, these assumptions are 
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Fig. 12 — Instantaneous phase distortion @(t) vs instantaneous frequency 
deviation w(t) of signal. 


permissible provided the rms phase error y defined by (12) and appearing 
in Fig. 4 is much less than 1. With these assumptions, the autocorrela- 
tion function of the output phase distortion is the same as for a square 
law device and is given by (Ref. 10, Equation 4.10-1) 


be [Ry (0) + 2Ry*(r)). (48) 


The first term can be identified with a de component that does not give 
rise to noise. The power spectrum of the nonlinear output phase distor- 
tion is obtained from the second component in (48) and is given by 


Wy? (w) = 2be? [ Rr cow unas (49) 
: 
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The ratio of average intermodulation noise power at the frequency w 
to the average signal power becomes 


1 
2b? i Rye Gy ceaceds 
0 





Wy (w) (50) 
p(w) = —2 3 = SS ,. 
Wy(w) kp(w)s(w)/w* 
In view of (46) the following relation applies 
Ry (r) = # | s(w) p(w) cos wr dw. . — (51) 
0 i 
Expression (50) can be written 
2,4 = —4 2 
ie 2bek i k Ry" (7) cos wr dr (52) 
if k?p(w)s(w) /o 
22 i: k Ry? (7) cos wr dr 
awe LPR Ome ig 
ple)s() | s(w)p(w) de 
where 
a= 0/2 = f/B (54) 
y = doy’ = BQ? = (2Qr)*be D’. (55) 
The following relation applies” 
[ RG eos ar= ; [ Weil Woo— awa: 266) 
0 0 
where Wy/(u) is the power spectrum given by (46). 
In view of (56) and (46), expression (53) can be written 
ary? 2 
pa) = ———St/ _ 
pw)sw) | p(w)s(w)de fixe 
[ s(w)p(w)s(w — u)p(w — udu. 
In the special case of a flat power spectrum s(w) = s of bandwidth 


* Ref. 10, Eq. (4C-6). In this reference the autocorrelation function is defined 
differently from the definition used here and has a factor 4 in integral (51), so that 
an additional factor } appears in (56). 
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Q = 27B, (57) becomes 


2 2/p? 1 Q 
ko) = p(u)p(w — u)du 
pla) 4 [' p(a)do® lea? oo 
ne selpwmaan. 69) 


p(a) [ p(x)dx “** 


When p(x) is of the form 
p(t) = 1 + e(u/Q)? = 1 + ec’ (60) 


relation (59) becomes 


7 30°? 1 ; : 
p(w) = “(1 + ca)(8 +) a G + ca) (1 + c(a = x) |dx 
ae (61) 
= y oa 
~ FOF eatV(8 Fa) OO 
where 
2.2 
F(ca) =2-a+ FF 4 1 — a) 
(62) 


ca 4 c 5 
—-Sl-G-aF+el+ (1 —a)i. 


In the particular case of pure FM, c = 0 and F(c,a) = 2 — a, so 
that (61) yields 


p(w) = re (2 — a) 


= (3) ya’ (2 — a) 


where a = w/Q = f/B, D = Q/2r and y = 62 = b.(2rD)’. 
The above result (63) conforms with an expression derived by Rice for 
this limiting case (Ref. 11, Equation 5.6). 


(63) 


A.5 Total Intermodulation from Quadratic Phase Distortion 


The previous analysis of the power spectrum of intermodulation noise 
was based on the assumption that the maximum phase distortion in the 
transmission band is substantially less than 180°. Without this limita- 
tion, numerical determination of the power spectrum becomes very diffi- 
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cult, though a formal solution may be feasible. However, it is possible to 
determine total intermodulation distortion without too much difficulty, 
without limitation on the phase distortion, as shown below. 

Let x designate the instantaneous amplitude of y’(t) = km/(t), and 
let x have a probability density 


3 
p(x) = (2) exp (—2'/20,”). (64) 


For large instantaneous frequency deviations y’(t) the derivative y’ (t) 
is on the average sufficiently small to be neglected. The total intermodu- 
lation distortion in the received signal prior to post-detection low-pass 
filtering is then for a nonlinear characteristic as illustrated in Fig. 12. 


fe [ “Gayo de = i: On be ode 


Ls Lom+1 (65) 
+ | (44 — box”) *p(a)dx +eee + (2am — box’) p(a)dx 
L3 Len-1 
where 
Lj = (j/bs)*. 
With box” = u, Y= boo,” and 
2 4 
plu) = (2) exp (—vé/2v) (66) 
TY 
expression (65) can be written 
i; iG 
fe [ pn dat (22 — w)’p(u)du 
0 1 
‘ (67) 
+f (40 — u’)"p(u)du + --- 
13 
where 
1; = (jr)? (68) 
Writing 2mr — wu? = —7, 2udu = dr, expression (67) can be trans- 
formed into 
Tv 3 | wT ae 
= aly 
i= i 7 p(r)dr + e [. Gr+n! $7) p(r)dr 
(69) 


co 2 
4. gly ie i p(r)dr +++: 
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where 


p(t) = (Gh) eter, (70) 


Total distortion J includes a mean or de power component J» that 
must be subtracted from J to obtain the nonlinear component. The mean 
amplitude component Jo’ Is given by 


ie [ “beatae | seam arenes 
4 (71) 
L5 
+ i (4 — box’) p(a)dx + ++: 


where L,, and p(a) are defined as before. 
With the same notation as before, (71) can be transformed into 


I = r'p(r)dr cg 22 he api - 7 p(r)dt 
(72) 


| x | 


Gear 


In the above relations y is the phase distortion corresponding to the 
rms frequency deviation as given by 


¥ = door = dQ? = dy. (73) 


+e" (r)dr +--+, 


The last relations follow from (29) since o,’ is the variance of y’(t). 
The total average signal power is 


s = nyo) = [0° [ Pao] /| plo)do] = at/0 (74) 


where C is a constant depending on p(w). 
The ratio of total nonlinear intermodulation noise to total average 
signal power becomes 





weit b_gitk 
S we 
(75) 


A.6 Total Intermodulation for Small Phase Distortion 
For sufficiently small values of y = 0.2", such that +/y > 1, only the 
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first integral in (69) needs to be considered. Hence 
/a~ / r'p(r)dr 
0 


= 37 erf (z) — 3-2¥y! exp(—z) — 2bay’ exp(2) (76) 
where 
g = n/2y. (77) 
With a similar approximation (72) yields 


Io & ; t’p(r)dr (78) 


= yerf (z) — 2? exp (—2’). 
For z 2 2, ory S 7/8: 
I 37 and Iv =y¥. 
Hence I — Ip) = 2y’ and (75) becomes 


2 


p= C2, (79) 


where the constant C is defined through (74). 

It will be noted that (79) is of the same basic form as (61) for the 
ratio p(w) at the frequency w. In (61) the multiplier of y’/z’ is a constant, 
as is the case in (79). 

A.7 Total Intermodulation for Large Phase Distortion 


When vy > 1, it is permissible to approximate p(r) as given by (70) 
with 


pr) ear (30) 


This approximation is valid in evaluation of the various integrals in (69) 
and (72) provided that for the minimum value of 7 = a, exp (—7/2y7) 
<1. This is the case if 


r/2y K1 or y> 7/2. 
With (80) in (69) 


c= e i fdr eo [tar |. (81) 
2ry 0 mal « (2am + 7)? 
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In (81), 
ie wdr 2 om + 1)4(32m2 — 8m + 3) 
_x (2mm + 7)3 a5 (82) 
— (2m + 1)3(32m + 8m + 3)] 
1 2) 
oie ar for m21. (83) 


For m = 1, (82) gives about 0.5 and (83) about 0.47. Hence (83) repre- 
sents a good approximation of (82). 
With (83) in (81) 


ry ig at < | 
= | — id = ; 4 
7 (..) | el + 3 » m3 (84) 
As a first approximation the summation can be replaced by an in- 
tegral, in which case 


PN 2 et Ba foe =a 
rw (sb) [ge +2 I. os 
With m = w 
3 
Jolbe82 Lal 
"Fart (4/2)| 
i ae; 7 
SH 
6 


for y>4r. (88) 





(86) 


bala 
: (cs 


sgh 
3 


By a similar approximation J 0 as given by (72) becomes 
Bo aay [? St omely f |r| dr | 
- (=) | 0 ot 2, : ~2 (2mr + 7)} v7 
_(1)[2 3 ied a 
e)[ere(¥9] 
4 
_va c*| ent (4/2) + i “| (91) 
y/ 373 


3 
for y>4r. (92) 


~ 
~~ 


dla 
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The ratio p is obtained from (75) with I as given by (87) and Iy' by 
(91). In the limit of y — © the ratio becomes 


oe et (7-1) 
e we ae 38. ¢6C«4 


C 
ear 


(93) 


A.8 Approximation for Total Intermodulation 
The general expression for the ratio p of total intermodulation noise 
power to average signal power can be written in the form 


o = Cay). (94) 
i) 


For the limiting case of y — 0, the function hf is in accordance with (79) 
h=y7. (95) 


For the other limiting case in which y — , the function h is in accord- 
ance with (93) 


h = 2.5/2 = 1.25. (96) 


In Fig. 18 are shown the above two limiting cases, together with the 
function h obtained from (75) as 7 = I — Io, when J and Jy are deter- 
mined from (86) and (91). The approximate function h(y) is obtained 
by drawing a transition curve between the above two limiting cases, as 
in Fig. 13. 


A.9 Approximation for Intermodulation Spectrum 


The function h(y) in Fig. 13 is proportional to the total intermodula- 
tion noise power and can be related to the power spectrum W,;(w) of 
intermodulation noise by 


h(y) = C is Wi(w) da (97) 


where ¢ is a constant. Relation (94) can thus be written 


20 [ Wilw) do. (98) 
we Jo 





p= 


For y — 0, (98) must conform with (95), which is possible provided 
the power spectrum is of the general form 
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Wo) = err? 5 Fo(w/2) (99) 


where /’p is any functional relation dependent only on the ratioa = w/Q. 
With (99) in (98) 


2 oe) 
Z Qcoc1C ua Fo(w/Q) dw 
ie Q Jo 


p — 
; . (100) 
= _ 2eueC | Fo(u) du. 
m 0 
This yields relation (95) provided 
ce | Folu) du = 1. (101) 
0 

















n(7) AND H(7) 
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Fig. 13 — Functions h(y) and H(y): 1, functions h(y) and H(y) for y < 1; 2, 
function h(y) for y > 1; 3, approximate "interpolated function h(y); 4, function 
H(y) for y > 1; 5, approximate interpolated function H(y). 
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From (100) it is apparent that the ratio of intermodulation noise 
power to average signal power in a narrow band dw at w is 


2 
1 
p(w) = 2 2eres0 = Fo(w/®). (102) 


Comparison of (102) with (61) shows that in this case 


3a° 


Ceara comer F(c,a) (103) 


il : 
2¢9C,C Q Fy(w/Q) = 
where F'(c,a) is given by (62). 
In summary, for y — 0 the power spectrum has a fixed shape inde- 
pendent of y and an amplitude proportional to 7’. 
Consider next the limiting case in which y > . In accordance with 


(96) h then approaches a constant, which is possible for various power 
spectra of the general form 


Ww) = Fo(w/y") (104) 


where F',,(w/y") is any functional relation dependent only on the ratio 
(w/y”"). In this case (104) in (98) yields 
2¢9¢1C 
we 


an ee | F(u) du 
B 0 





I Fala/y") deo as 





where u = w/y". 

The exponent can be determined from consideration of the input vs 
output characteristic shown in Fig. 12. If be is increased by a factor k, 
the intervals between zero points are multiplied by a factor k™?, as indi- 
cated in Fig. 14 for k = 4. For a given frequency deviation, the band- 
width of the power spectrum is then multiplied by a factor k’ and the 
amplitude of the spectrum at each frequency multiplied by a factor 
k7*, Hence in the case of quadratic phase distortion as considered here, 
n = 4 in (104). 

Based on the above considerations, the power spectrum at any fre- 
quency w for the above two limiting cases would vary with y as indicated 
in Fig. 13. The shape of the curves between these two limiting cases 
would in a first approximation be represented by the function H(7) 
shown in Fig. 13. 
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Fig. 14 — (a) Relation of instantaneous phase distortion @(t) to instantaneous 
frequency deviation u(é) for a given bz; (6) relation of instantaneous phase dis- 
tortion to instantaneous frequency deviation with fourfold increase in be . 


A.10 Approximation for p(w) 


The ratio p(w) of intermodulation noise power in a narrow band at w 
to average signal power in the same narrow band can be written 


20a) 


p(w) = H(y). (106) 





This relation differs from (94) in that h(y) as shown in Fig. 13 is re- 
placed by H(v) shown in the same figure, and C is replaced by C(w). 
The constant C defined through (74) depends on the frequency pre- 
emphasis p(w). The function C(w) depends both on the frequency pre- 
emphasis p(w) and the frequency under consideration. 

For the particular type of frequency pre-emphasis represented by 


9 
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(60), expression (106) must conform with (61). This results in the fol- 
lowing approximate relation 


os (#) Gea) H(y) (107) 


where H(y) is the function shown in Fig. 13 and 
3a” 
(1 + ca?)(8 + c) 


where F'(c,a) is given by (62). 
In the particular case in which c = 16 anda = f/B = 1 


G(c,a) = F(c,a) (108) 





G(c,a) & 0.192 (109) 
and (107) yields 
RB 2 
p(B) = (2) X 0.192H (y) (110) 
B 2 
= (2) X 0.192-y° for y¥K1. (111) 
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Cutoff Frequencies of the Dielectrically 
Loaded Comb Structure as Used in 
Traveling-Wave Masers* 


By S. E. HARRIS, R. W. DEGRASSE and E. O. SCHULZ-DUBOIS 
(Manuscript received June 27, 1963) 


The subject of traveling-wave maser design is reviewed and a first step 
towards an analytical design procedure is presented. A method is derived 
for calculating the upper and lower cutoff frequencies of a comb-type slow- 
wave structure of simple geometry. It ts based on the electromagnetic field 
pattern and the equivalent impedances which are calculated for these fre- 
quencies, both for the dielectrically loaded and the empty comb structure. 
The design procedure resulting from these calculations permits the predic- 
tion of a dielectric loading geometry that shifts the upper and lower cutoff 
frequency of the empty comb to new, lower values which can be arbitrarily 
specified within certain limitations. Frequencies calculated by this pro- 
cedure are compared with the results of measurements, and it is found that 
cutoff frequencies can be predicted to better than 10 per cent. 


I. INTRODUCTION 


In the early development of the traveling-wave maser (TWM),! the 
design procedures used were largely empirical. Short TWM model sec- 
tions were built, tested and modified in order to meet the desired per- 
formance specifications. By this cut-and-try method, a satisfactory de- 
sign was finally derived which was applied in the construction of 
full-length TWM’s. 

However, a more satisfying approach is possible if the relevant the- 
oretical aspects regarding the maser active material, the ferrimagnetic 
isolator and the electromagnetic behavior of the slow-wave structure 
are known, either rigorously or approximately. Then a TWM can be 
designed on the basis of analysis before actual construction. Most at- 
tractive in the analytical approach is the inherent flexibility and versa- 


* This work was supported in part by the U.S. Army Signal Corps under Con- 
tract DA 36-039-se-85357. 
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tility. Thus, a large number of design ideas may be explored and a near 
optimum configuration can be found before any hardware is built. 

The present paper is a step in the direction of a more analytical ap- 
proach. Using reasonably accurate approximations to the field pattern 
at both cutoff frequencies, the equivalent TEM line impedances, the 
“effective” dielectric constants and, finally, the cutoff frequencies are 
calculated. This results in a numerical design procedure for the TWM 
structure. The analysis is made for a comb having fingers of rectangular 
cross section and for dielectric loading with maser material in the form 
of one or two rectangular parallelepipeds as shown in Fig. 1. Compari- 
son of cutoff frequencies calculated by this method with experiment 
shows agreement to usually better than 5 per cent. 
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Fig. 1 — Typical comb structure. 
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1.1 The Significance of Cutoff Frequencies in TWM Design 
Consider the TWM electronic gain formula! 
G(db) = 27.3(— x” )fFl/0, . (1) 


Here (— x”) is the inverted susceptibility of the maser active material, 
f the signal frequency, F the filling factor, / the length of the maser 
structure and v, the group velocity. The TWM net gain is obtained by 
subtracting from (1) the slow-wave structure loss (copper loss) and the 
ferrimagnetic isolator loss (ferrite loss). 

In the development of a practical TWM, the design frequency f and 
the structure length / are generally determined by the application. The 
susceptibility (— x”) is a property of the active material which cannot 
be theoretically predicted and must be experimentally determined. 
(—x”) is redefined as 


— x” — Ix0” (2) 


and the quantities J and xo” are determined by two independent measure- 
ments. Here, J is the inversion ratio, i.e., the ratio of electronic gain 
from the activated maser material to electronic loss in the same material 
at. thermal equilibrium. J: E. Geusic and W. J. Tabor have carried out 
inversion measurements for ruby maser material in a helix test structure, 
and the method and results will be described in a forthcoming paper.’ 
The susceptibility at thermal equilibrium, xo”, is measured by standard 
resonance techniques’ or may be calculated from the material composi- 
tion and linewidth. In this way, —x” can be determined to about 10 
per cent, which is adequate for the present design procedure. Complica- 
tions can arise in practice, however, if nominally identical crystals show 
variations in the active ion concentration or in the crystalline perfection. 
The filling factor F may be factorized into two expressions 


F = F,F, (3) 


Fo=|flenttaa]/[iat f ia taa| (4) 
re=| fp iwpaa]/| fie taa]. (5) 


Here, u is the magnetic dipole moment associated with the maser signal 


where 


and 
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transition and H is the RF magnetic field in the TWM structure. The 
asterisk * denotes the conjugate complex time dependence. The integra- 
tion is performed in the cross-sectional plane where J/ denotes the cross 
section of the maser material and A the total structure cross section. 
F,, may be called the polarization efficiency factor and F, the volume 
filling factor. Ff, expresses the excitation efficiency of the signal transi- 
tion by the RF magnetic field present in the maser material. For example, 
if both » and H are of circular polarization in the same direction, then 
F, is unity. Similarly, for maser material symmetrically loaded on both 
sides of the comb and with a circular transition perpendicular to the 
finger direction, a symmetry argument shows that ’, = %. F, indicates 
what fraction of the total magnetic field energy is contained within the 
maser material. F, and F, are functions of frequency across the pass- 
band of the comb structure. Usually, however, it is sufficient to consider 
F at some midband frequency where it is only a slowly varying function 
of frequency. 

Experience suggests that it is possible to estimate F to fair accuracy 
from the TWM geometry and a qualitative estimate of the RF magnetic . 
field pattern. For example, it is estimated that in TWM’s designed in 
this laboratory for 5.6, 4.2, 2.4 and 1.4 ge the filling factor F varies over 
the relatively limited range from 25 to 45 per cent. Thus, from the view- 
point of the analytical design of the TWM, a detailed computation of 
the RF magnetic field configuration is of no great value unless the other 
factors entering the TWM gain formula are known with comparable 
accuracy. 

Up to the present time, this was not the case, the factor least amenable 
to analytical prediction being the group velocity v, . It is well known 
that a wave traveling through a slow-wave structure has field com- 
ponents varying like exp [7(wt — 8z)], where w = 2zf, ¢ is the time, 8 
the phase propagation constant and z the length coordinate along the 
structure. In the comb structure, each finger is an energy storage element 
capable of resonant storage in the same way as a quarter-wavelength 
coaxial resonator. As a general rule, the phase shift between adjacent 
elements may assume values between 0 and -+:z as the frequency is 
varied across the passband. The phase shift values 0 and +a are as- 
sociated with the cutoff frequencies. The comb structure is normally a 
forward-wave structure, where +7 is the phase shift at the upper cutoff 
frequency and 0 that at the lower. It is possible (although not of practical 
importance in TWM design) to make the comb a backward-wave 
structure, in which case —7z is the phase shift at the lower cutoff fre- 
quency and 0 that at the upper. In the normal forward-wave comb 
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structure, the phase propagation constant then varies from 6 = 0 to 
8 = (N — 1)z/l across the passband; where N is the number of fingers 
and / is the structure length measured between centers of the first and 
last finger. The group velocity is given by 


v, = dw/dp. (6) 


Typical diagrams of 8 as a function of w are shown in Fig. 2(a). As the 
curves approach the cutoff points, they assume infinite slope, correspond- 
ing to zero group velocity. There is a range at midband, however, where 
the group velocity is fairly constant. These graphs are typical of most of 
the structures studied but exceptions occasionally were found, as in- 
dicated in Fig. 2(b). These exceptions include backward-wave structures 
where phase and group propagation take place in opposite directions. 
They also include “mongrel” structures where, over part of the band, 
6 is a double-valued function of w; these, therefore, are forward and 
backward at the same time. This latter case is a very undesirable one; 
as discussed in Ref. 4, the existence of two propagation modes at the 
same frequency, one a forward wave, the other a backward wave, allows 
for propagation with gain in both directions despite the presence of an 
isolator. As a result, the maser will oscillate instead of offering stable 
gain. Empirically, however, this situation can be easily diagnosed and 
there are remedies to rectify it. Therefore, double-valued w-6 relations 
may be excluded from the present considerations. 

With this proviso, it can be seen from Fig. 2(a) that the midband 
group velocity can be estimated reasonably well from a knowledge of 
the two cutoff frequencies alone, viz. 


Vo = 2aAfl/(N — 1) = 2aAfAl. (7a) 


Here Af is the frequency width of the passband, Al is the center-to-center 
spacing between comb fingers and a is a numerical factor which takes 
into account the detailed shape of the w-8 curve. Equation (7a) may be 
rearranged in terms of the group velocity slowing 


e 1d/2 f 
Saal Ay ae) 
indicating that slowing is partly a geometric effect, i.e., the compression 
of a half wavelength into one period of the structure, and partly the ef- 
fect of compression in the frequency domain, sometimes expressed by a 
loaded Q. a@ assumes values of one for a straight line w-6 relation, 1.57 
for an inverse cosine, and may in practice be as high as four for a 
“sagging” w-8 curve. In other words, the uncertainty in estimating the 
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Fig. 2 —(a)Typical forward w-8 diagrams of loaded comb structures with nor- 
malized cutoff frequencies. (b) Exceptional w-8 diagrams found in comb structures 
with extreme dielectric loading. 
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group velocity or slowing from the cutoff frequencies is not very large, 
usually less than a factor of two. 

Thus, it is clear that a method for calculating the two cutoff frequencies 
would be an important first step towards an analytical design procedure. 
Such a mathematical method should be carried out as rigorously as 
possible. The reason for this may be demonstrated in the following way. 
If fringe capacity at the finger tips and dielectric loading effects are 
neglected, the comb structure is electrically equivalent to the Easitron”™ 
structure. In this approximation, the comb would be a zero passband 
structure with identical cutoff frequencies like the Easitron. In reality, 
they differ only because fringe capacity and dielectric loading affect 
both frequencies to different degrees. Thus, the width of the passband 
Af is obtained as a small difference between large numbers, the upper 
and lower cutoff frequencies, fy and f, . To obtain Af with fair accuracy, 
ju and f, must be known with good accuracy. Similarly, a small change 
in the dielectric loading may change fy and f, each by a small percentage, 
but Af by an appreciable factor. Experience has shown that comb struc- 
tures with different dielectric loadings may have a passband width Af 
anywhere between 1 and 50 per cent of the midband frequency. In other 
words, as long as the cutoff frequencies are not known, the uncertainty 
in an estimate of v, may be almost two orders of magnitude. The compu- 
tation of the cutoff frequencies would be very useful if it could reduce 
this uncertainty to about a factor of two. Besides determining the group 
velocity and hence, indirectly the electronic gain, the cutoff frequencies 
also define the center frequency and the tunable bandwidth of the TWM. 
Since it is impossible to matcha structure right up to the cutoff frequency, 
the useful tunable band is well inside the structure passband Af. An 
analytical design procedure that allows a reasonably accurate prediction 
of the cutoff frequencies would clearly be desirable, as center frequency 
and tunable bandwidth are among the primary TWM specifications. 


1.2 The Function of Slow-Wave Structures in Electron Beam Tubes and 
TWM’s . 


A considerable amount of work, both theoretical and experimental, 
has gone into the study of slow-wave structures for tubes. It would be 
gratifying if this knowledge could be used in TWM work. Unfortunately, 


* The Easitron was analyzed by L. R. Walker, unpublished manuscript, quoted 
in Ref, 1. This structure consists of a rectangular waveguide with an array of uni- 
form, identical conductors inthe H planeconnecting both short walls. It has zero 
passband, nonpropagating resonances of frequencies where the conductor length 
is one or more half wavelengths. 
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this work has only limited applicability to the TWM. This is more readily 
understood if slow-wave structures for electron beam tubes and’ for 
TWM’s are compared. 

In tubes, slowing factors between 10 and 100 are typical, while in the 
TWM, slowing of 50 to 1000 is used. This difference influences primarily 
the mechanical tolerances, which are tighter for higher slowing. 

A more fundamental distinction concerns the applicable slowing con- 
cept. In an electron beam tube there must be synchronism between the 
electromagnetic mode propagated on the slow-wave structure and the 
interacting mode characterized by a charge distribution on the beam. 
Therefore, the analysis of tubes is concerned with the phase velocity of 
the slow-wave structure mode. Similarly, in a traveling-wave parametric 
amplifier there must be synchronism between pump, idler and signal 
propagation, requiring a phase velocity relation for these three fre- 
quencies. In filter circuits the condition of synchronism is usually satis- 
fied only over a small fraction of the total structure bandwidth. By 
contrast, the amplification by the maser material does not depend on the 
existence of phase relations along the TWM structure. The maser 
material may be considered as an incoherent, long-time energy reservoir 
from which energy is withdrawn upon stimulation by an incident signal 
and added to the incident signal in a coherent phase preserving fashion. 
The function of the slowing is merely to “give the signal more time”’ to 
interact with the energy stored in the maser material, i.e., to enhance 
the stimulating: gain interaction. Thus, the analysis of TWM’s is con- 
cerned with the signal: group velocity in the structure rather than phase 
velocity. It is not necessary that v, be constant over the tunable band. 
If the gain over the tunable band is required to be constant, then the 
product —x’Ff/v, (neglecting copper and ferrite losses) should be con- 
stant over the band. Experience has shown that this condition can be 
met over almost the entire passband. 

Another point is the interaction mechanism between the active ele- 
ment and the slowing structure. An electron beam interacts with a struc- 
ture mode via the RF electric field, and the interaction is conven- 
tionally represented by an interaction impedance. The interaction of the 
inverted spins in the maser material with the structure mode takes place 
via the RF magnetic field, and its strength is measured by the filling 
factor. - 

. All the differences mentioned have no bearing on the question whether 
the knowledge of slow-wave structures accumulated in studies directed 
towards electron beam interaction can be applied to TWM structures. 
For example, the degree of slowing is not essential for a theoretical anal- 
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ysis, the group slowing is easily derived by differentiation from the phase 
propagation, and electric and magnetic interaction terms can be obtained 
equally well from the field analysis. 

The chief difference in slow-wave structures for these two applications 
lies in their relation to dielectric loading. In a tube, dielectric loading is 
undesirable and is usually avoided as far as possible. By virtue of its 
dielectric constant, the glass envelope of a TWT, for example, drags 
away from the beam some of the electric field energy carried by the helix 
and thus reduces the gain interaction. In fact, most studies of slow-wave 
structures for beam tubes pertain to metal structures surrounded by 
vacuum. 

Dielectric loading, being an undesirable side effect for tubes, is an 
essential and rather beneficial feature in maser structures. Since the gain 
interaction is magnetic in nature, the interaction of the electric field 
with dielectrics may be used to advantage without deteriorating the gain 
interaction. Indeed, it is being used for reducing the over-all maser size, 
tuning the band center frequency, adjusting the tunable bandwidth or 
increasing the gain by increased slowing (of course, the items mentioned 
are not independent). Thus, a high degree of design flexibility can be ob- 
tained, even with the identical copper comb, merely by changing the di- 
electric loading. 

For this reason, dielectric loading must be included in any treatment 
of TWM structures. The present paper is a first contribution to the the- 
oretical treatment of maser structures taking dielectric loading into ac- 
count. To keep the mathematics reasonably simple, the maser comb ge- 
ometry, including the dielectric loading, was chosen to be fairly simple. 
In the laboratory, dielectric loading techniques were developed in which 
the loading consists of more than one dielectric and has more complex 
shapes. Work to be published by F. S. Chen has generalized the analysis 
to take these modifications into account. It also expands the present 
analysis of the cutoff frequencies into a more general one which allows 
the prediction of the entire w-8 diagram. This will be particularly valua- 
ble in finding criteria to avoid structures having a double-valued ‘‘fold- 
over” or “mongrel” w-8 diagram. 


II. GENERAL PROBLEM AND APPROACH TO SOLUTION 


The problem is to find by analysis the upper and lower cutoff fre- 
quencies of the comb-type slow-wave structure as used in a traveling- 
wave maser (TWM). In particular, this implies taking into account the 
dielectric effect resulting from loading the comb with maser material or 
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possibly some other dielectric material and the effect of the fringe ca- 
pacity at the tips of the comb finger. It was pointed out before that, in a 
zero-order approximation neglecting both effects, the comb is a zero 
passband structure. 

In the course of this treatment it will be necessary to introduce a num- 
ber of restrictions and approximations. These are mostly required in 
order to keep the mathematics manageable. Some other restrictions are 
introduced in order to have the geometry underlying the calculations 
correspond to the type of TWM geometry which is presently investi- 
gated in the laboratory. These various restrictions and approximations 
are labeled with lower-case roman numerals for reference in this discus- 
sion. 

(4) The first restriction pertains to the cross section of the comb fingers. 
The treatment used here is applicable only to combs with fingers of rectangu- 
lar cross section. 

This means that it is not possible to apply this type of analysis to a 
comb having round fingers as used in the original TWM’s. It may be 
mentioned here, however, that it is possible to treat the round-finger 
comb as long as certain simple frequency or impedance data are available 
from measurements on scale models, resistance cards or measurements 
in the electrolytic tank. 

Besides being better suited for mathematical analysis, there is another 
justification for treating combs with rectangular fingers. This has to do 
with fabrication of combs. There is indication that it is possible to 
fabricate combs with rectangular fingers not only with greater ease but 
also with greater perfection. The subject of these fabrication techniques 
may be discussed at some later date. 

A typical comb structure as treated here is shown in Fig. 1. The fingers 
shown are of square cross section and are spaced by a finger width. It 
should be emphasized that the general method used here is applicable 
to any rectangular cross section and spacing, although a great many of 
the computations are concerned with square fingers spaced by a finger 
width. 

(it) The next restriction is that maser material (or some other dielectric) 
is inserted into the comb in the shape of a single rectangular parallelepiped. 

The restriction to parallelepipeds is rather definite. There is a possi- 
bility, however, of considering more than one slab of maser material 
loading the comb. No change in the general analysis is required if two 
identical slabs are considered which are loaded symmetrically on both 
sides of the comb. This is shown in Fig. 3(a). The analysis could be carried 
out also for the case shown in Fig. 3(b) where the maser material is in- 





Fig. 3 — Loading geometries. 


serted in the form of two pairs of identical slabs. It should be mentioned, 
however, that the calculation will be appreciably more cumbersome in 
this case. Although it will not be described in detail, it will be fairly ob- 
vious to the reader how the calculations have to be modified to take into 
account geometries like the one of Fig. 3(b). 

(it) A further simplifying assumption is that the dielectric loading is 
assumed to have an isotropic dielectric constant, at least for field components 
perpendicular to the finger direction. 

This assumption is not too restrictive. An effective dielectric constant 
may be estimated in the case of a tensor dielectric constant. This estimate 
will usually be different for either cutoff frequency, since it depends on 
the electric field configuration. As the tensor components are always of 
the same order of magnitude, the estimated effective dielectric constant 
should turn out to be sufficiently accurate for most cases of practical 
interest. 

No provisions have to be made for magnetic permeability. Outside the 
maser signal line, pn = 1 for the maser material. Even within the fre- 
quency range of the signal line, the deviation of » from unity is so small 
that it can be neglected for all practical purposes as a factor influencing 
the cutoff frequencies. A similar reasoning applies to the ferrimagnetic 
isolator. Even though the values of u’ — 1 are larger there, they are less 
effective due to the very small ferrimagnetic filling factor. 

The starting point for the calculation is the phase shift. At one cutoff 
frequency the phase shift between fingers is zero. This has the conse- 
quence that an instantaneous electric field pattern within the comb may 
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look like Fig. 4(a). Usually, although not necessarily so, this is the case 
at the lower cutoff frequency, f, . Throughout the paper this case will be 
referred to as the “ower cutoff,” although the term “zero phase shift 
case” would be more appropriate. The field pattern is repetitive and 
shows no field lines from finger to finger since they are on the same po- 
tential. It is symmetric with respect to a cross-sectional plane in the 
structure which contains either the center line of a finger or the center 
line in the space between two fingers. Therefore the same field pattern is 
obtained with a single finger if the section of the comb containing this 
finger is enclosed by a ‘‘magnetic wall.” A magnetic wall is a fictitious 
plane on which the electromagnetic field components obey boundary 
conditions such that the electric field is tangential and the magnetic field 
normal to the plane. These boundary conditions are opposite from those 
on a perfect conductor. The perfect conductor is closely approximated 
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Fig. 4— Field patterns showing phase shift conditions. 
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in experiments by high-conductivity metals, whereas the magnetic wall 
is a mathematical model only. Since the field patterns of Figs. 4(a) and 
4(c) are identical, the frequencies will be the same, too. Thus the lower 
cutoff frequency of the comb can be found as the resonant frequency of 
the one-finger structure in Fig. 4(c). 

A similar reasoning applies to the upper cutoff frequency fy . Here the 
phase shift is between adjacent fingers. An instantaneous field pattern 
will therefore look like Fig. 4(b). Since adjacent fingers are subject to 
opposite potential, there are strong electric field components going from 
finger to finger. The field pattern is symmetric with-respect to a cross- 
sectional plane in the structure which contains the center line of a finger, 
but antisymmetric with respect to a cross-sectional plane which contains 
the center line in the space between two fingers. Thus the same field 
pattern can be realized on a single finger if the section of the comb con- 
taining the finger is enclosed by a perfectly conducting (or metallic) wall. 
This wall will take the place of the plane of antisymmetry in the comb. 
This is illustrated in Fig. 4(d). Again, identical field patterns require the 
same frequency. Thus the upper cutoff frequency of the comb can be 
found as the resonant frequency of the one-finger structure in Fig. 4(d). 

The method of determining the resonance frequency of either one-finger 
model, that of Fig. 4(c) or 4(d), is suggested by Fig. 5. The finger acts 
essentially as a quarter-wave TEM resonator. At the comb base, this 
TEM line is terminated in a short. At the finger tip the TEM line is 
terminated by a nearly perfect “open.” This is only slightly modified 
by fringing electric fields between the finger tip and the surrounding 
walls. The effect of these fields can be lumped into a fringe capacity C. 
In principle, C will be different for both cutoff frequencies. 

Unfortunately, both capacities Cy and C’, cannot be calculated easily. 
Therefore, measurements have been made in an analog electrolytic tank 
setup. A scale model having the cross section of the one-finger lines in 
Figs. 4(c) and 4(d) was built. This cross section is shown in Fig. 6(a) for 
the upper cutoff frequency and in Fig. 6(b) for the lower. In the elec- 
trolytic tank the electric field lines of the object under study are simu- 
lated by the current lines in the tank fluid. No approximation is involved 
in this analogy. In particular, it is possible to simulate a magnetic wall 
like that of Fig. 3(c) by an insulating wall. This is done in the cross sec- 
tion used in the lower cutoff analog measurement shown in Fig. 6(b). 
In the analog measurements, the metal configuration was first lowered to 
the insulating bottom of the tank as indicated in Fig. 6(c). The resistance 
measured between electrodes in this fashion is proportional to the im- 
pedance of the corresponding TEM mode of the one-finger line; it is 
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Fig. 5 — Equivalent one-finger line: (a) geometry considered, (b) two-wire line model, (c) simplified equivalent 
TEM mode line model; use either subscript U for upper, or L for lower, cutoff. 
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inversely proportional to the capacitance of the line. For a second meas- 
urement, the finger was raised to the proper scaled height and a metal 
plate was placed on the bottom of the tank. The inverse of the resistance 
so measured is proportional to the capacity of the appropriate length of 
one-finger TEM line plus the fringe:tapacity arising from the diverging 
field pattern beyond the end of the finger. [See Fig. 6(d).] The conduc- 
tivity of the tap water used was measured also. From these measure- 
ments it is possible then to evaluate the fringe capacity as well as im- 
pedance and ‘capacitance of the TEM mode on the one-finger line. 

In Fig. 7, the fringe capacitance values C', for the lower cutoff fre- 
quency and Cy for the upper cutoff frequency obtained from the tank 
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Fig. 7 — Fringing capacitance Cz for lower cutoff frequency and Cy for upper 
cutoff frequency. The geometry of the comb used includes fingers of cross section 
0.040 x 0.040 inch, spaced 0.080 inch on center in a housing 0.240 inch wide (ratio 
Wv/Dy = 1.25). Capacitance is plotted vs spacing d between finger tips and the 
opposing housing wall. 
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measurements are shown as a function of the distance d between the 
finger tips and the opposite waveguide wall. The data are valid for fingers 
of square cross section, Dy/2 X Dyu/2 = 0.040 X 0.040 inch, spaced 
center-to-center by Dy = 0.080 inch and contained in a housing of width 
2Wu + Dz/2 = 0.240 inch (aspect ratio Wy/Dy = 1.25). The Wy and 
Dy are the dimensions of the empty comb, as shown in Fig. 9 in Section 
III. It should be mentioned here that these data can be applied to 
dimensions other than those indicated if one observes two facts. First, 
if all linear dimensions are scaled simultaneously by some factor, the 
capacity is scaled by the same factor. Second, experience has shown that 
the fringe capacity is a very slow function of the ratio Wu/Du ; no 
noticeable errors were found when these capacity values were used for 
Wv/Dy values ranging from 0.75 to 1.5. 

Another experimental method to determine the fringe capacity is 
based on the availability of either a comb structure of exact size or a 
scale model. The upper and lower cutoff frequencies, fzyv and frx , of the 
empty, unloaded structure are measured by direct measurement. The 
result of the measurement can best be expressed in terms of a new ef- 
fective finger length, Ly or L,. This is based on the fact that a trans- 
mission line of physical length L’, shorted at one end and terminated 
with a small capacity at the other, is electrically equivalent to a some- 
what longer transmission line which is shorted at one end and open at 
the other. The effective finger lengths are different for both cutoff fre- 
quencies 


Cc 








Ly = a =L + Aly (8a) 
Ly = a L +Al,. (8b) 


Here c is the velocity of light. 

The relation between these length dimensions and the fringe capacity 
involves the characteristic impedance of the line. The fringe capacity 
follows from 


, 
a = Zev tan otal = Zev cot etfavAlu (9a) 
2rfeuvCu c Cc 
and 
! 
_1i = Ley tan mie Zar cob eee, (9b) 
2a fatCr Cc Cc 
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For the particular structure geometry investigated here in detail, the 
fringe capacity was determined using these equations and the character- 
istic impedances derived later in this section. The values of C, and Cy 
obtained agree well with these from the tank measurement. 

For the subsequent calculations it is assumed that the new effective 
lengths, Ly and Lz , are known. If, instead, the fringe capacities, Cy and 
C,, are known, the new effective lengths can be calculated using the 
impedances Z rv and Zz . Since the capacities are small, (9a) and (9b) 
can be approximated by 


Aly = ZevCvc (10a) 
Aly = ZrrC re. (10b) 


In this fashion, the problem of Fig. 5(b) is reduced to that of Fig. 5(c). 
The cutoff frequencies, f, and fy , are found as resonance frequencies of 
a transmission line L; or Ly long, where one end is shorted, the other 
open, and a length / is partially loaded with dielectric. 

The field pattern in the unloaded part of the transmission line is rigor- 
ously a TEM mode.. Therefore, the impedance of this line can be found 
by a resistance card or an electrolytic tank technique. The electrodes are 
shaped for the model in the same way as the conductors in the unloaded 
TEM line. Then the impedance of the line is simply equal to the re- 
sistance measured in the model provided the resistance per square is 
adjusted to or scaled to 377 ohms. In addition to this measuring tech- 
nique, these impedances, Zry and Zzz, will be determined analytically 
below. This involves a calculation with good accuracy of the electric field 
pattern. 

The dielectrically loaded section of the transmission line would, if 
treated with the same rigor, require a much more involved procedure. 
Therefore, at this point an approximation is introduced. 

(tv) The field configuration in the loaded part of the transmission line 
can be treated asa TEM mode. 

In reality, this is not true. An exact solution of Maxwell’s equations 
for a TEM-type transmission line having a cross section partly filled 
with dielectric is not a TEM mode. Instead, the process of matching 
boundary conditions requires the presence of longitudinal field com- 
ponents. It can be seen, however, that these longitudinal components will 
become smaller with decreasing frequency and vanish in the zero fre- 
quency limit. Thus this approximation implies the representation of a 
dynamic field configuration by its static analog. The accuracy of such 
an approximation, therefore, tends to be better the shorter the linear 
dimensions involved are with respect to the wavelength. In the range of 
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dimensions used here it is expected that no appreciable loss of accuracy 
is incurred in this connection. 

The consequences of treating the field configuration in the loaded part 
of the one-finger model as a TEM mode are far-reaching and very helpful 
for the subsequent analysis. Considering the same metal boundaries as 
in the unloaded part, the field configuration in the loaded part has to be 
the same. This follows from the fact that the TEM fields are given as a 
unique solution to Laplace’s equation for the appropriate geometry. 
Thus one way to treat the loaded part of the one-finger model, consistent 
with a TEM mode in the same geometry, is by an effective dielectric 
constant. This allows for a reformulation of approximation (iv): 

The part of the transmission line loaded partially by a high dielectric 
constant material can be treated as if it were loaded uniformly throughout 
the cross section with a material of a lower “effective” dielectric constant. 

This effective dielectric constant will, of course, be different for the 
upper and lower cutoff frequencies. Using these effective dielectric con- 
stants, éy and é,, the impedances and propagation constants of the 
loaded section are related to those of the empty section by 


Zor = Zei/Va, Zou = Zav/V ev (11) 
Bor = Vix Bar Bou = Véu Bev. (12) 


Here the first indices FE and D refer to the empty and dielectrically 
loaded line, the second indices Z and U to the lower and upper cutoff 
frequencies. The propagation constants in the empty TEM line are, of 
course, identical to that in vacuum 
Ber = (2nf1/c) Bev = (2nfv/c). (13) 
Assuming for the moment that the effective finger lengths, Dy and Ly , 
the characteristic impedances of the empty line, Zry and Zzz, and the 
effective dielectric constants, éy and é, , are known, the cutoff frequencies, 
fu and f,, can be calculated. The procedure is to match voltage and 
current at the boundary between the loaded and unloaded section of 
the line. This results in impedance equations 


Zru cot Beu(Ly —= l) = Zov tan Boul (14a) 
Ze cot Bar( Ly = l) = Zou tan Boul. (14b) 


These are rewritten in @ more convenient form 
— T = l fu T l ) fe 
a pom — re eee Duis = fee Saas 1] 
Vey = tan k Vey te] tan k (1 ae (15a) 
= L ft. l ) f “| 
é = _ Se | 15b 
Vet tan | Ve, — Ts fe|t an E ( be fat ( 5 ) 
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These equations are identical for lower and upper cutoff frequencies. 
They do not contain the characteristic impedances explicitly. PnOy are 
solved in the following way. 

~/éy or ~/é, is considered a given parameter. Then the frequency 
ratio fu/fev or fx/fzris a function of l/Lz or l/Ly . This function requires 
the solution of transcendental equation (15a) or (15b). Numerical 
values were obtained by machine computations using the IBM 7090. 
The results are plotted in Fig. 8. 

This graph can then be used to determine the upper and lower cutoff 
frequencies of the loaded comb structure. It is assumed here that the 
upper and lower cutoff frequencies of the empty comb, fev and fr, , and 
connected: with them, the effective finger lengths, Ly and L,, are known. 
They are best determined by measurement, but they could also be cal- 
culated from the fringe capacity and the characteristic impedance. The 
quantity yet to be evaluated is the effective dielectric constant, éy and 
é,, before the cutoff frequencies can be read from the graph in Fig. 8. 
It will be necessary, however, to work out the electric field pattern within 
the unloaded comb, then in the loaded section, including the respective 
characteristic impedances, before the effective dielectric constant can be 
obtained. 


III, FIELD PATTERN AND CHARACTERISTIC IMPEDANCE OF UNLOADED COMB 


3.1 Upper Cutoff Frequency 


The electric field pattern of the unloaded one-finger model will look 
about like Fig. 9(a). This geometry is, unfortunately, too complicated 
for a closed analytical treatment. On the basis of the geometry and the 
mathematical tools at hand, the following approach may be suggested. 
The area available to the electric field is divided into four regions, two 
equivalent regions of type A and two equivalent regions of type B, as 
shown in Fig. 9(b). Two further approximations are then necessary. 

(v) The electric field in the regions A can be represented as a homogene- 
ous, parallel plate condenser field. 

(vt) The electric field in the regions B can be represented by the field pro- 
duced by an infinitely thin metal fin inserted in a rectangular enclosure of 
corresponding dimensions. 

These approximations are illustrated in Figs. 9(c) and 9(d). Along the 
joints of regions A and B the field thus assumed is discontinuous. In 
reality, it is inhomogeneous near the boundary of region A, and it is 
less inhomogeneous than assumed near the boundary of region A because 
there is only a 90° bend, not a 180° bend as in the model used. These 
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Fig. 8 — Plot of numerical values obtained by machine computations. 


discrepancies of the field model from what would be expected should in 
reality be very small, particularly if the gap between finger and wall, the 
dimension W4 = 34(1 — r)Dy defined in Figs. 9(a) and 9(e), is small 
compared to other dimensions. This is so in cases of practical interest. 

As far as the impedance is concerned, the two regions A and the two 
regions B are in parallel. The impedance of a region A is simply the ratio 
of its dimensions multiplied by free-space impedance. The impedance of 
region B is not as easily found. It is possible, however, to use a conformal 
transformation which maps the region B into a parallel plate geometry. 
This is schematically indicated in Fig. 9(e). The transformation actually 
utilized consists of the consecutive application of two transformations, 
each using elliptical functions. The procedure, including the mathemati- 
cal details of the conformal transformation by elliptical functions, is 
outlined in the Appendix. 

It is known from the theory of conformal mapping by functions of 
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Fig. 9 — Analysis for upper cutoff frequency: (a) real field patterns, ny re- 
gions rained for analysis, (¢) homogeneous field assumed in region A, (d) fin field 
assumed in region B, (e) fin field equivalent to homogeneous field. 


complex variables that the geometry is preserved in infinitesimal regions. 
In particular, it is clear that infinitesimal squares with boundaries 
formed by field lines and equipotential lines continue to be squares. 
Since the impedance can be thought of as composed of the impedance of 
these infinitesimal squares, partly in parallel and partly in series as in- 
dicated by the over-all geometry, it follows finally that the impedance 
of the two transmission lines of Fig. 9(e) is the same. 

The geometry before transformation is characterized by the two ratios: 
Wy/Dv and r. Thus W,’/D,’ will be a function of both of these ratios. 
So far only combs with r = 4 have been investigated in practice. For 
convenience, therefore, the subsequent calculations are carried out for 
this value of 7. This implies a further restriction. 


TWM COMB STRUCTURE CUTOFF FREQUENCIES 459 


(vit) In the numerical calculations to follow, only comb geometries with 
the finger width as large as the gap between fingers are considered. 

From a mathematical point of view, this restriction is somewhat 
arbitrary. Any other choice of 7, the ratio of finger width to length of 
period, however, would necessitate another application of the elliptic 
integral conformal transformation. 

With r = 4, Wo'/Dy’ is a single-valued function of Wy/Dy. This 
function is plotted in Fig. 10, An interesting feature of this graph is that 
Wv'/Dy' goes asymptotically to 4; it reaches this value to within 2 per 
cent at Wy/Dy = 0.65. The physical interpretation of this observation 
is as follows. For Wy/Dy > 0.65, essentially all the field lines originating 
at the center fin in Fig. 9(d) terminate on the side wall; none reach the 
opposite end wall. Therefore, this wall can be moved out toward infinity 
with no noticeable effect on the impedance at the upper cutoff frequency. 

The characteristic impedance of the empty structure at the upper 
cutoff frequency can now be given. It is 


- Di x-9 DE 
Lou = 377 ohms / (2 W, + Z a) . (16a) 


An important special case is one where, first, Wu/Dvy is greater than 
0.65 so that the asymptotic value Wy’/Dy’ = 4 applies and where, 
second, the fingers have a square cross section so that W4/Da = 3}. 


Then the characteristic impedance is simply 
Zou = + X 377 ohms = 47.1 ohms. (16b) 
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Fig. 10 — Conformal transformation for upper cutoff. 
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‘Since the partial impedances are equal, it also follows in this case 
that the total stored energy is equally distributed between the four 
regions A; A, B, B. This remark may be helpful in estimating the filling 
factor. 


3.2 Lower Cutoff Frequency 


The procedure here is quite similar to that in the case of the upper 
cutoff frequency. The field pattern is illustrated in Fig. 11(a). The 
cross-section area available to the electric field is divided into four re- 
gions, two electrically equivalent regions of type A and two regions of 
type B, as shown in Fig. 11(b). Again two approximations are required. 

(viit) The electric, field in region A ts so small that tt can be neglected. 


—_— 


MAPPING 











(C) (d) 


Fig. 11 — Analysis for lower cutoff frequency: (a) typical electric field pattern, 
(b) regions used for analysis, region A assumed field-free, (c) fin field assumed in 
region B, (d) fin field equivalent to homogeneous field. 
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(12) The electric field in region B can be represented by the field produced 
by an infinitely thin metal fin inserted into a rectangular enclosure with ap- 
propriate dimensions and boundary conditions. 

It is apparent that the approximation (viii) is justified. Only very 
small fringing fields will exist in region A. The implication of approxima- 
tion (7x) is indicated in Fig. 11(c). It should also be very well justified, 
since there is no essential difference between the idealized field pattern 
and the real one. Region B can be transformed into a simple parallel 
plate geometry. This is indicated in Fig. 11(d). The transformation 
again consists of two consecutive conformal mappings by means of el- 
liptic functions. The procedure is outlined in the Appendix. The imped- 
ance of region B is simply given by the aspect ratio W,//D,' of the 
parallel plate geometry resulting from the transformation, multiplied by 
the free-space impedance. This resulting ratio W,’/D1’ is a function of 
two ratios, rand W/D. For mathematical convenience and because of 
practical importance, only comb geometries with r = 14 are considered 
in the subsequent calculations. For other ratios 7, a new evaluation of 
the elliptical transformation is necessary. Thus restriction (viz) is in- 
voked here, too. 

(vit) In the numerical calculations which follow, only comb geometries 
with the finger width equal to the gap width between fingers are considered. 

The single-valued function W,’/Dz1' of W1/D, with the parameter 
r = 4 is shown in Fig. 12. The characteristic impedance of the empty 
structure at the lower cutoff frequency is then given by 





/ 
Zo = ap X 377 ohms. (17a) 
As long as W,/D, > 0.2, it is seen from the graph that this can be 
approximated by 
Tipe (F Et 0.11) 377 ohms. (17b) 
2\D. 

The asymptotically linear curve in Fig. 12 and this last equation suggest 
an almost obvious interpretation. The electrical behavior of region B is 
essentially the same as that of a parallel plate geometry having the same 
width D,;’ = D,z, buta slightly greater distance between plates, Wz’ > 
W,. Also, the asymptotic slope for the curve is unity. Considering a 
geometry with W,/D1 > 0.2, this would mean the following. If Wz is 
increased further, the electric field pattern near the fin stays the same, 
while the added volume away from the finger is taken up by a homo- 

geneous electric field. 


462 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 














































a 
Pane ceeeeeeS 




















4. 05 06 OF 08 O9 tO uo 
Wt Wp 








0 04 O02 03 0 2 1.3 1.4 1.5 


Fig. 12 — Conformal transformation for lower cutoff. 


IV. CAPACITANCE AND EFFECTIVE DIELECTRIC CONSTANT OF COMB PAR- 
TIALLY FILLED WITH DIELECTRIC 


It was mentioned that the electromagnetic field configuration in the 
comb line partially loaded with dielectric should be treated as a THM 
mode. It was pointed out that this is equivalent to finding a static solu- 
tion of the electric field problem. Thus the problem here is to find the 
static value of the capacitance per unit length of the loaded finger line. 
The difference in electrical behavior of the loaded line compared to the 
unloaded line is then fully expressed by an effective dielectric constant. 
This effective dielectric constant is simply the ratio of the static capaci- 
tance of the loaded line to the capacitance of the unloaded line. 
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4.1 Upper Cutoff Frequency 


The field pattern in the presence of one dielectric slab is illustrated in 
Fig. 13. It is seen that the dielectric is present in one of the regions called 
B before. The usual boundary conditions for the continuity of the tan- 
gential L vector and of the normal D vector have to be observed in 
fitting together the electric field pattern inside and outside the dielectric. 

At first sight it seems that no difficulty is incurred in this respect at 
the boundary of the dielectric. In the model chosen for the field con- 
figuration, the field lines run parallel to the boundary both in regions A 
and B. The boundary condition for tangential electric field seems to ap- 
ply, with the consequence that the field pattern remains the same in the 
dielectric as before in the unloaded region B. Calculations are based on 
this assumption, and they are presumably of sufficient accuracy for 
present purposes. 

There is a small error in this assumption. It was pointed out before 
that the two models chosen to represent the field in regions A and B do 
not match at the boundary. In the models, the field in A is homogeneous, 
that in B strongly inhomogeneous. The real field at the boundary of A 
and B should be somewhere between these two extremes. It is expected, 





Fig. 13 — Electric field pattern at upper cutoff with dielectric loading present. 
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therefore, that the error in the impedance calculation of the empty comb 
at the upper cutoff frequency is negligible. The same is not necessarily 
true in the presence of dielectric. The real, inhomogeneous field in the 
region near the boundary of A and B will be disturbed by the insertion of 
dielectric. The deviations of the real field from that used in the calcula- 
tions — homogeneous in A, elliptic function field in B— are now ac- 
centuated by a high dielectric constant rather than evened out as in the 
empty comb. This will lead to an error in the calculation of the capaci- 
tance and the effective dielectric constant. Hence it is not trivial that 
the approximations (v) and (vz) are still reasonably good in the presence 
of dielectric. Fortunately, it can be argued that the error incurred by this 
approximation is still negligible within the accuracy sought for here and 
with respect to typical structure geometries and dielectric constants 
considered. A formulation of the approximation follows. 

(x) In the presence of dielectric loading, the static electric field can still 
be represented by a homogeneous, parallel plate field in region A and the 
field of a metal fin inside a rectangular enclosure in region B filled by the 
dielectric. 

The next concern is the other boundary of the dielectric away from 
the finger. Here the field lines cross the boundary at all directions be- 
tween tangential and perpendicular. It would be very difficult to apply 
boundary conditions to this field pattern. Therefore another restriction 
is introduced. 

(xt) The calculation is restricted to dielectric loadings thick enough so 
that essentially the total electric field energy of region B 1s contained within 
the dielectric. 

The numerical implication of this restriction follows directly from 
Fig. 10. It is assumed that the fingers are as wide as the gap between 
them. From the graph the following fact can be deduced. If a: geometry 
is considered where Wy is considerably larger than Dy , then 98 per cent 
of the electric field energy is concentrated in a rectangle near the finger, 
Dy wide and 0.65 Dy deep. Restriction (a7) thus implies that only di- 
electric slabs which have a thickness of at least 0.65 times the length of a 
period of the comb are considered. 

Fortunately, this restriction does not exclude any cases of practical 
interest. Since the field configuration on the finger is treated here as a 
TEM mode, the filling factor in the plane perpendicular to the finger 
is the same for the dielectric and the magnetic field energy. Thus, slabs 
thinner than indicated by restriction (xz) would also have a reduced gain 
interaction near the upper cutoff frequency, since not all of the magnetic 
field energy of region B would be contained in the maser material. Gain 
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is still at a premium in present TWM development, and thus it does not 
seem to be necessary to treat cases other than those restricted by (27). 

It is now possible to write down the capacitance and the effective di- 
electric constant. By comparison with (16a), it is seen that the capaci- 
tance per unit length of the empty one-finger line is 


D Dy’ 
cov = | 27 + 2 e | (18) 
(Lower case c is used to distinguish this quantity from the fringe ca- 
pacity Cy.) With dielectric loading on one side of the finger 


cou =f 224 + (et 1) PE | oe (19a) 
Similarly, if the dielectric is loaded on both sides of the finger 
cou = «| 2 PA 262] oo, (19b) 
The effective dielectric constant is then simply, for loading on one side 
év = [(e + 1) + 26]/[2 + 20] / (20a) 
and for loading on both sides 
éy = (e + b)/(1 +b) (20b) 
with 
b = DsaWy’/W4Do'. (21) 


Most important perhaps for present applications is the case where, first, 
the fingers are square so that (16b) applies and where, second, the di- 
electric is ruby with an isotropic average dielectric constant of « = 9. 
In that case, for loading on one side 


éy = 3 (22a) 
and for loading on both sides 


(22b) 


m 

q 
I 

On 


4.2 Lower Cutoff Frequency 


The field pattern in the presence of one dielectric slab is illustrated in 
Fig. 14. The dielectric fills part of the region called B before. For the 
evaluation of the capacitance it is significant that restriction (a7) is 
applied here, too. Then the following approximation can be made. 
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Fig. 14 — Electric field pattern at lower cutoff with dielectric loading present. 


(xii) In the presence of dielectric loading, the static electric field can be 
represented in the following way: There is zero field in region A; in the 
dielectric there 1s a field like that produced by a metal fin in a rectangular 
enclosure, having the dimensions of the dielectric and subject to appropriate 
boundary conditions. The field past the dielectric is a homogeneous parallel 
plate field. 

It can be argued that these approximations are well justified. There 
is no potential difference between fingers; hence region. A should be field- 
free except perhaps for some very small fringe fields. In connection with 
17(b) it was shown that the field has its inhomogeneities near the finger, 
whereas the field region near the wall is reasonably homogeneous. 

The capacitance per unit length of the loaded one-finger model can 
now be given. For the empty line it is 


Cer = 2e(D,'/W,1’). (23) 


For the loaded line, the capacitance is obtained from two contributions 
in parallel, one from each side of the finger. The capacitance of the loaded 
side comes from two contributions in series: one from the dielectric, in- 
volving an elliptical transformation using the dimensions of the dielectric, 
and one a parallel plate contribution from the space behind the dielectric. 
Thus, for dielectric loading on one side 
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_.;, fede W\ 
CoL = €0 E 7 + (Se, + Ws) | (24a) 
and for loading on both sides 
2 » , We 
CoL = 2 | (Ge, + vs) . (24b) 


Here Wp and D> are the physical dimensions of the dielectric cross sec- 
tion per one-finger line, Wz and Dy, the dimensions of the empty space 
behind the dielectric. Wp’/Dp’ is obtained from Wp/Dp by means of the 
elliptical transformation illustrated in Fig. 11. 

The effective dielectric constant can now be evaluated as the ratio 
Cyt/Cz,. The formulas, however, turn out to be fairly long. They are 
given here, therefore, only for the case that the approximation in (17b) 
is valid both for the empty structure and the dielectric. It is further 
observed that 


D, = Dp = Dz = D 
and 
Wi= Wot We. 
Then the effective dielectric constant for dielectric loading on one side is 


ye — —s — 1)Wp + (e + 1)0.11D 


2 —(e—1)W, +0.11D (25a) 
and similarly, for loading on both sides 
bce W,+ 0.11D (25b) 


eWr — (e —1)Wn + 0.11D° 


It is seen that the effective dielectric constant is a function of «, Wo/W 1 
and D/W, . Once a particular structure geometry has been picked, then 
D/W, is known. If a particular maser material is selected, ¢ is known. 
Then é, is a unique function of the relative loading thickness, Wp/W 1. 
One example of such a function is given in Fig. 15. For convenience in 
using the graph of Fig. 8, the square root ~+/é, is given instead of é,. 
Curves for effective constants based on other parameters can easily be 
calculated using either (25a) or (25b). 


V. EXAMPLE FOR DESIGN PROCEDURE 


In Sections III and IV the empty and the dielectrically loaded comb 
structure were evaluated. Field pattern, impedance and propagation 
constants were obtained for both the upper and lower cutoff frequencies. 
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Fig. 15 — Vz vs relative loading thickness, one side only, loaded at lower 
cutoff frequency for parameters indicated. 


With this information at hand, it is now possible to arrive at a numerical 
design procedure. The aim is to predict the cross-sectional dimensions of 
a dielectric parallelepiped which will simultaneously tune the upper and 
lower cutoff frequencies of the comb structure to some predetermined 
values. Of course, it is not possible to ask for completely arbitrary design 
cutoff frequencies. Obviously there are limits to the amount of tuning 
which can be achieved by a given dielectric material within a given comb 
geometry. These limits can also be determined easily by the analysis. 

The design procedure follows the outlines given briefly at the end of 
Section IT. It can now be described in general terms. Perhaps it is ad- 
vantageous, however, to illustrate the procedure by means of an ex- 
ample. The example to be described is a case of a ‘“‘design on paper.” 
That is to say, the design calculations can be made entirely on the basis 
of calculable values. It is not necessary to fabricate a size or scale model 
of the comb structure under consideration in order to determine certain 
values by measurement. The only empirical value required is the fringe 
capacity between finger tip and the structure enclosure; this may be ob- 
tained from Fig. 7. 

One interesting and valuable feature of the design procedure is that of 
independently setting the upper and lower cutoff frequencies. This is 
possible because the upper cutoff frequency can be controlled by adjust- 
ing the height / of the dielectric loading alone, and because it is not de- 
pendent in any way on the dielectric thickness Wp as long as Wp ex- 
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ceeds a certain small minimum value. Then the dimension Wp p can be 
used to control the lower cutoff frequency independently. 

As an example for the design procedure, a comb structure is considered 
with the following dimensions: 

(a) finger length 0.400 inch 

(b) spacing between fingers 0.040 inch 

(c) finger cross section 0.040 square inch 

(d) wall-to-wall spacing of enclosure 0.240 inch. 

As further information, the fringe capacity was measured in an electro- 
lytic tank model and was found to be (see Fig. 7 for gap spacing greater 
than 70 mils): 

(e) fringe capacity C, = 0.025 puF, Cu = 0.035 wyF. 

The problem considered is that of finding the dimensions for a single 
ruby parallelepiped which brings the upper cutoff frequency to 4200 me 
and the lower cutoff frequency to 3210 me. 

First step: Find effective finger length at the upper cutoff frequency of 
the empty comb. 

Equation (10a) applies for the increase in length and (16b) applies 
for the impedance; thus 


Aly = ZeyCuc 
= 47.1 X 0.025 X 10°" X 3 x 10” 
= 0.035 cm. 
The effective length for upper cutoff is then [see (8a)] 
Ly = L' + Aly 


= 2.54 X 0.400 + 0.035 = 1.051 cm. 


This corresponds to an upper cutoff frequency for the empty comb [see 
(8a)] 

is EU = c/ 4Ly 

= 7150 me. 

Thus the design specification 

fu = 4200 me 
is equivalent to specifying a ratio of 

fu/feu = 9.587. 


Second step: Find in an analogous way the effective finger length at 
the lower cutoff frequency of the empty comb. 
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Equation (17b) applies for the impedance. I'rom the dimensions given, 
W, = 0.100”, Di, = 0.080”, hence 


_1/W, 
Vier => 3 (fe + 0.11) 377 ohms 


256 ohms. 
The addition to length is given by (10b) 
Al, = ZriC 1c 
256 X 0.035 x 10°" x 3 X 10" 
= 0.268 cm. 


ll 


The effective length for lower cutoff becomes 
DL, =L' + Aly 
= 0.400 X 2.54 + 0.268 
= 1.284 em 


corresponding to a cutoff frequency for the empty comb 


fer = c/4Ly 
= 5840 mc. 
The design specification of 
fx = 3210 mec 


is thus equivalent to specifying a ratio 


fi/fer oe 0.55. 


Third step: Satisfy the upper cutoff frequency specification by choos- 
ing an appropriate dielectric height / without regard for Wp , the thick- 
ness of the loading. This is possible because, as mentioned before, the 
effective dielectric constant at the upper cutoff frequency is independent 
of loading thickness. The effective dielectric constant, év , for one-sided 
loading with ruby is 3 from (22a); thus 


Vey = 1.73. 


Consulting Fig. 8 for the dielectric height which makes fu/fzyv = 0.59 
with the parameter ~/é, , it is seen that 


l/Ly = 0.96. 
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Hence, the dielectric loading height should be 
l = 0.96 X 1.051 = 1.010 cm 
0.398” 


In other words, the dielectric loading height turns out to be very nearly 
the same as the finger length. 

Fourth step: Satisfy the lower cutoff frequency specification by choos- 
ing an appropriate thickness Wp of the dielectric loading. This is done 
by the following successive measures. 

From the loading height / just determined find 


1/L, = 1.010/1.284 
= 0.79. 


Enter the graph of Fig. 8 with 1/L, = 0.79 and f1/far = 0.55. The 
value interpolated at the point having these two coordinates is 


n/a, = 2.22. 


The graph of Fig. 15 is valid for present calculations; entering this last 
value into the graph it is found that 


Wrn=W,1 


ll 


hence 
W> = 0.100 inch. 


The final answer, then, is that the comb described initially will have 
the specified cutoff frequencies if a slab of ruby of height 0.398 inch and 
of width 0.100 inch is inserted. 

An experiment was carried out to check the results of this calculation. 
The two cutoff frequencies of a comb as specified above were measured 
after inserting a single slab of polycrystalline high density alumina (di- 
electric constant 9.3) with cross-sectional dimensions of 0.400 inch and 
0.100 inch. The cutoff frequencies measured were 4200 me and 3210 me 
respectively. These frequencies were then specified as design frequencies 
for the above example. The close agreement between the actual dimen- 
sions of the alumina slab and those calculated by the present recipe is 
gratifying. It may be argued, however, that the obtained agreement is 
somewhat fortuitous. In particular, one should expect that the fringe 
capacity is altered if the dielectric loading extends all the way along the 
fingers up to the finger tips. To investigate the accuracy of the present 
analysis, a series of systematic measurements was made. 
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For this study a number of short sections of comb structures were 
built and tested. They all had finger dimensions of 0.040 « 0.040 x 
0.445 inch, and the fingers were spaced 0.080 inch on center. The struc- 
tures were loaded symmetrically with two slabs of high-density poly- 
crystalline alumina (dielectric constant quoted to be 9.3) of full finger 
height. The geometry and the result of the measurements are shown in 
Fig. 16. In two series of measurements, the fraction of the housing width 
filled by the alumina loading, Wp/Wz,z, was held at 0.90 and 0.95, re- 
spectively, while the gap width between the finger and the housing wall, 
W, = Wz, was varied in the range 0.75D = 0.060 inch, D = 0.080 inch, 
1.25D = 0.100 inch and 1.5D = 0.120 inch. From the analysis, it is 
known that fy should be independent of these dimensional changes. This 
is borne out by the experiment. Both the experimental points and the 
solid line for the theoretical value of fy show the frequency independence. 
It is observed, however, that the experimental frequencies are 3.5 per 
cent higher. A somewhat greater disagreement is found for the lower cut- 
off frequency, which seems to indicate a systematic trend between theory 
and experiment. It can be said, however, that the largest deviations are 
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Fig. 16 — IXxamples of measured and calculated cutoff frequencies; the insert 
shows the comb geometry used. 
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10 per cent and that the typical discrepancy between theory and experi- 
ment is less than 5 per cent. The chance for greater systematic errors in- 
creases, of course, if comb and loading geometries are considered which 
comply less rigorously with the restrictions and approximations made in 
the text. 

The numerical examples shown demonstrate that dielectric loading in- 
deed decreases the fundamental passband frequency of the empty comb 
by a very appreciable factor. A one-sided loading with ruby may reduce 
the frequencies by a factor of 1.7, while double-sided loading may lead 
to a reduction by a factor 2.5. Still greater reductions may be obtained 
by using dielectric materials with higher dielectric constants and by 
modified comb geometries, in particular by changing the finger cross 
section from square to rectangular. It is also clear from this treatment 
that the shaping of the dielectric loading can be used to vary the degree 
of slowing within wide limits. These remarks may suffice here to illus- 
trate the prominent role of dielectric loading techniques in the field of 
TWM development which was pointed out in the Introduction. 

Since the original derivation of this analysis in 1960, several TWM’s 
have been developed in this laboratory. They include the TWM for the 
ground station receiver in the Telstar satellite communication experi- 
ment‘ and radio astronomy TWM preamplifiers for hydrogen line work 
at 1420 me.’ In these cases, the analysis has proved to be a valuable aid 
for arriving at a first-order design and similarly for providing guidelines 
in the subsequent improvements of these designs. 


APPENDIX 


The conformal mapping transformations are derived and evaluated, 
leading to the impedance transformation curves in Figs. 10 and 12. The 
mathematical treatment given here is not too extensive, because the 
type of transformation used is known from other areas of electrical en- 
gineering. Yet the description of the mathematical procedure is made 
reasonably complete so that it may be useful as a guide for treating other 
related problems: for example, traveling-wave masers where the finger 
width is not identical to the spacing between fingers. 


A.1 The Schwartz-Christoffel Transformation 


The particular conformal transformation used here is a special case of 
the more general Schwartz-Christoffel transformation. The theorem 
proved independently by these two mathematicians states that it is possi- 
ble to find an analytical function which maps the inside of a polygon on the 
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complex plane into the upper half of this plane. The boundary of the 
polygon thus is mapped into the real axis. If two transformations are con- 
sidered, one of the type mentioned, the other performing the inverse 
function, it follows that the inside of a polygon can be mapped into the 
inside of any other polygon. 

The general Schwartz-Christoffel transformation is illustrated in Fig. 
17. For purposes of discussion, it is perhaps easier to consider first the 
inverse transformation of the upper half of the complex plane into a 
polygon. The transformation will be accomplished by a function whose 
derivative is given by a product of the type 


RS ts (w _ a)! (a _ b) FF (w _ oc)? nie (26) 


VK 
W-PLANE 


4 
10] 





Fig. 17 — Illustration of the general mapping properties of the Schwartz- 
Christoffel transformation. 
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To demonstrate the transformation property, consider values of w and 
dw on the real axis, Also represent each factor in the form r,e*** with a 
real number 7, for the magnitude and ¢, for the angle. It is seen then 
that for values w such that w > a,b,c --- all the ¢, on the right-hand 
side of (26) vanish. Hence the angles of dz and dw are identical; that is, 
these line elements are parallel. Mathematically 


Adz = 0 if w and dw are realand w > a,b,c ---. (27a) 


For values a < w < b the first bracket changes sign; that is, its angle 
is r. The angle of the first factor becomes a — 


Adz? =a-— qT if w and dw’ are realanda > w> be---. (27b) 


That is to say, the real axis of the w plane near a is transformed in the 
z plane into a polygon corner at some as yet undetermined point A 
including an angle a. Similarly 


Adz” =a-+6— 2 if wand dw” are realanda,b > w>c--- (27c) 


indicating another polygon corner at B including an angle 6 and cor- 
responding to the point b on the real axis of the w plane. 

In this fashion, it is shown that the transformation (26) indeed maps 

the upper half of the w plane into the inside of a polygon having speci- 
fied angles a, 8, y --- at points in the z plane corresponding to a,b,c --- 
in the w plane. While it is thus easy to satisfy conditions on the angles 
of the polygon, the difficulty is to find the points A,B,C --- in the z plane 
which correspond to a,b,c --- in the w plane. This requires an evaluation 
of the integral of (26). 
F Even more typical for engineering applications, and important in the 
present example, is the inverse situation. The corner points A,B,C --- 
of the polygon are given, Then the problem is to find the real numbers 
a,b,c --+ which when inserted into (26) will transform this polygon into 
the upper half of the w plane. In most cases, this problem can only be 
solved numerically. The procedure would be to tabulate integrals of (26) 
for some range of values a,b,c --- . Numbering such tables with the 
given integral values A,B,C ---, the appropriate transformation 
parameters a,b,c could be picked. 

To keep the need for tabulation down to a manageable chore, the 
number of significant parameters has to be restricted as much as possible. 
The example of importance in this connection is the mapping of a rec- 
tangle into the upper half of the complex plane. The number of significant 
parameters here can be reduced to one, the length ratio of two adjacent 
sides. Other parameters can be eliminated by trivial transformations 
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such as scaling and rotation of the coordinate system, which is ac- 
complished simultaneously by a complex constant factor in (26) or a 
shift of the coordinate origin which corresponds to the integration con- 
stant of (26). 


A.2 Mapping of a Rectangle into the Upper Half of the Complex Plane 


It is now possible to write down the transformation equation for a 
rectangle. The conventional notation is illustrated in Tig. 18. The 
corners of the rectangle in the z plane are the complex numbers K, 
K + 1K’, —K + 7K’ and —K. In the w plane they correspond to the 
points 1, 1/k, —1/k and —1 on the real axis. 


/ 
W- PLANE 





Fig. 18 — Illustration of the transformation of a rectangle in the z plane into 
the upper half of the w plane, introducing the conventional mathematical nota- 
tion. 
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From (26) the transformation derivative is 


de = Alw- *) (w — 1) Kw +1)? (w + t) . (28) 
dw k k 
When the constant A is chosen appropriately (A = —1/k) this becomes 
dw 
% = (wid — Pa el 
and 
as dw 
an i (1 — w*)#(1 — kw)?” (30) 


This integral is an elliptical integral of the first kind. It, gives z as a 
function of w and k, where k is referred to as the modulus of the integral. 
From the definition adapted in the figures it follows that 


Kx [ 7 ES (31) 
and 
1/4 
iK' = es ES ea (32) 
K is called the complete elliptical integral. A’ is the complete integral to 
the complementary modulus obeying the functional relationship 
K'(k) = K(k’) (33) 


where k” + k” = 1 isused to definethe modulus k’ ascomplementary to k. 

The definition of the elliptical integral of the first kind as given in 
in (30) is due to Jacobi. Many tables use also the notation of Legendre. 
This is obtained by setting 


w=sing, dw = cos¢d¢ 
(34) 
k = sin 8, k’ = cos 8. 
Then 
¢ 
dy 
7 he (1 — sin? 6 sin? W) (35) 
a{2 
dv 
A> jo = amt an wi (36) 
a{2 
oe dv 
|. aera (37) 
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From this discussion it is clear that the transformation of a rectangle 
into the upper half plane requires finding the modulus k or equivalently 
the modular angle 6 of the elliptical integral from the given geometry of 
the rectangle. It is further clear that K and K’ are not independent, but 
related through either (31) and (32) or (86) and (87). Therefore, it is 
not possible to specify both length dimensions of the rectangle of Fig. 18 
but rather only their ratio. The problem thus is reduced to finding the 
dependence of the modulus k or 6 from the aspect ratio K’/2K of the 
rectangle. 

This functional dependence was evaluated using the Smithsonian 
Elliptic Function Tables, in particular tables of complete elliptical 
integrals. The result is presented in Fig. 19. 

It should be added that frequently, instead of the elliptical integral 
(30), its inverse is used. This inverse function is written 


w = snz modulo k (38) 


which is defined to mean (30). This notation is reminiscent of the sine 
function, with which the sn function is indeed identical in the special 
case k = 0. 


A.3 Mapping of the Upper Cutoff Frequency Configuration 


It is now possible to carry out the mapping transformations used in 
the comb structure analysis. The initial geometry for the lower cutoff 
frequency is indicated in Fig. 20(a), where solid lines represent conduct- 
ing electrodes. The final result is a parallel plate geometry like that of 
Fig. 20(d). This figure represents the cross section of an idealized trans- 
mission line for which the impedance is simply given by the ratio of the 
length dimensions times free-space impedance. The transformation makes 
use of two intermediate steps. The interior of the rectangle (Fig. 20a) 
is first mapped into the upper half of the complex plane (Fig. 20b). 
Then a readjustment of the scale leads to Fig. 20(c). Then the upper 
half plane is finally mapped into the inside of the desired rectangle 
(Fig. 20d) with electrodes only on opposite sides. 

To keep track of these steps, the relevant points in the original geome- 
try and their transforms are denoted by capital letters O,A,B --- . The 
first and second elliptical transformations are distinguished by indices 1 
and 2 attached to the modulus and the complete integral values. The 
mapping then proceeds as follows. 

(a) From the z plane to the y plane. 


y = sn z modulo k; = sin (39) 
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Fig. 19 — Relation between the ratio of the length dimensions of the rectangle 
to be transformed and the modular angle of the transforming elliptical function. 


The modular angle 6, is found by entering the curves of Fig. 19 with the 
aspect ratio W/D = K,’/2K) of the original rectangle. The corresponding 
coordinates in the z and y planes are given in Table I. The transformation 
of points O through D requires only the graph of Fig. 19. For points A 
and F, use has to be made of elliptic function tables. In the Smithsonian 
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TABLE J — SUMMARY OF TRANSFORMATIONS 
FOR Uprer Curorr FREQUENCY CASE 








3 y x Ww 
O 0 0 0 0 
B Ky ] 1/(sn Kir mod ky) Keo + tKe’ 
E —Ki ; —1 —1/(sn Kir mod k1) —K.,+ 71K.’ 
a = - a Poi not of interest 
A Kur sn Kir mod k; 1 Ke 
EF —Kir —sn Kir mod ky -1 —Ko 

ky sin 61 


ll 


ko = sin 02 
62 = @ = sin-! [sn Kir mod fk, or 4;] 


Tables the Legendre notation (34), (35), (86), and (37) is used. Entering 
these tables with 2 = Ky and the angle 6,, a value of ¢ in radians is 
found. This value ¢ is converted to degrees and renamed 62. 

(b) From the y plane to the x plane. 

This is a change of scale and is accomplished by dividing all values by 


sin ¢@ = k, = sn Kir mod 6,. (40) 


After this step the arrangement of the points OBEAF on the real axis 
is the standard one for transformation of the upper half plane into a 
rectangle. 

(c) From the x plane to the w plane. 

This transformation finally shapes the original electrode geometry 
into the desired parallel plane geometry. The transformation is indi- 
cated in Table I. However, since the interest centers only on the im- 
pedance — that is, the length-dimension ratio of this final rectangle — it 
is not necessary to carry out this transformation in detail. This ratio 
W’'/D’ = K,'/2K2 is obtained from Fig. 19 by entering it with the 
modular angle 6. = @ = sin ky. 

Following these steps in the case r = 4, the curve of Fig. 10 was ob- 
tained. 

A short-cut is possible if 6, < 30°; that is, if W/D > 0.65. In that 
case the sn function can be approximated by a sine function and K ~& 
a/2. Then @ = 6 = r/2; in particular, for r = 3, @ = 0 = 45° and 
W'/D’ = 3. 


A.4 Mapping of the Lower Cutoff Frequency Configuration 


The procedure is quite similar to that used for the upper cutoff fre- 
quency geometry. It is summarized in Table II and Fig. 21. 
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TaBLE I] — SuMMARY oF TRANSFORMATIONS 
FoR LOWER Curorr FREQUENCY CASE 





Z y x w 
O 0 0 0 0 
: BS a not of interest 
C Ky + 1K, 1/ky V/(ki sn Kur mod ky) Ke 4- 1K 2’ 
D —K, + Ky’ —I/ki —1/(ki sn Kir mod ki)| —Ke + tK2’ 
A rKy sn Kir mod ky 1 Ke 
F —rky —sn Kyr mod ky; —] —Ke 
ky = sin 6; 
ke = sin 62 
62 = sin-! [(sn Kir X sin 6;) mod &; or 4) 


(a) From the z plane to the y plane. 

This step is identical to the first transformation of the upper cutoff 
frequency configuration. 

(b) From the y plane to the x plane. 

This scaling is also the same as that used before. The difference is, 
however, that now the points C and D are of interest, whereas before the 
points considered were B and E. 

(c) From the x plane to the w plane. 

Here the transformation differs; now a different modulus 


ky = ky sn Kyr mod ky 


is used. The resulting complete integral values K, and K,.’ are not to be 
confused with those obtained for the upper cutoff frequency case. Since 
the interest centers only on the impedance value K,’/2K. = W’/D’ of the 
resulting rectangle, it is not necessary to evaluate this transformation in 
detail. The numerical evaluation is quite similar to the one of the upper 
cutoff situation. Using Fig. 19, one finds the first modular angle @, from 
Ki /2K, = W/D of the original geometry. Entering the tables with z = 
Kyr and 6, , an integral value ¢ is found. This value is obtained in radians. 
Then form 


II 


k. = sin 6. = sin 6; X (sn Kyr mod hk) 


(41) 


II 


sin 6; X (sin ¢@ mod 9@;). 


Using this formula, the angle 6, is evaluated in degrees. Then the graphs 
(Fig. 19) can be used again to obtain from 6, the length dimension ratio 
W'/D’ of the transformed rectangle. 

Following this procedure for the case r = 3, the graph of Fig. 12 was 
obtained. 
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Permutation Decoding of 
Systematic Codes 


By JESSIE MACWILLIAMS 
(Manuscript received September 4, 1963) 


A symmetry of a systematic code is a permutation of bit positions in each 
code word (the same permutation is applied to all code words) which pre- 
serves the code as a whole. Permutation decoding makes use of these sym- 
metries to build up a decoding algorithm for the code. 

It is difficult to find an appropriate set of symmetries for a code picked 
at random. For cyclic codes the problem is somewhat easier, and for some 
special cyclic codes it 1s solved completely in this paper. For these codes, at 
least, 1t 1s evident that permutation decoding is easy to implement and in- 
expensive compared with other decoding schemes. 

Permutation decoding as a reans of error control ts evaluated for the 
binary symmetric channel and for the switched telephone network as repre- 
sented by experimental data. It 1s found to be extremely effective on the binary 
symmetric channel and of very doubiful value on the present telephone net- 
work. 


INTRODUCTION 


A systematic code of block length 7 is a subspace of the vector space 
of all possible rows of n symbols chosen from a finite field. In this paper 
such a code will be called an alphabet,’ and the sequences belonging to 
the alphabet will be called letters. 

The parameters used to describe an dishabek are block length, n, num- 
ber of information places, k, and error correcting capability, e: n is the 
number of symbols in each letter, k is the dimension of the alphabet as a 
vector space, and e is defined by the property that the minimum Ham- 
ming distance between two letters is either 2e + 1 or 2e + 2. 

It is well known! that an alphabet with parameters n,k,e is theoretically 
capable of correcting all occurrences of Se errors in a block of length n. 
However for e > 1, the process of error correction by decoding is com- 
plicated, and likely to require expensive equipment. In this paper we 
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describe a new decoding scheme, permutation decoding, which is con- 
ceptually simple and quite easy to implement. 

The decoding procedure consists of a sequence of permutations of the 
received block of symbols, each of which is followed _by a parity check 
calculation. We can thus make a rough comparison between the com- 
plexity of the equipment required for encoding and decoding. The en- 
coder uses one parity check register, and the decoder uses r (or uses one 
r times), wherer is the number of permutations in the decoding sequence. 
Real time operation with a constant time delay is possible and perhaps 
not too expensive. 

Permutation decoding owes much to the previous work of Peter Neu- 
mann? and Eugene Prange.’ It depends essentially on the symmetries of 
the alphabet. A symmetry of an alphabet means a permutation of digit 
positions which preserves the alphabet as a whole. The same permuta- 
tion is applied to the digits of every letter, and each letter is changed, if 
at all, into another letter of the same alphabet. Very little is known about 
symmetries of alphabets in general, but it will be shown that even this 
little is enough to enable us to apply the decoding scheme to a large class 
of alphabets. 

Permutation decoding differs from previous schemes in two important 
ways. First, it becomes easier as the redundancy of the alphabet in- 
creases; it is most useful for alphabets with high error correcting capa- 
bilities. Secondly, it cannot correct more than e errors in n places. A 
received sequence containing more than ¢ errors either will be ‘‘corrected”’ 
wrongly or will emerge unchanged from the decoder. 

It will become apparent in Section III that permutation decoding pro- 
duces many more undetected errors than does error control by detection 
and retransmission. The simpler scheme of detection and retransmission 
should be used when it is at all feasible. 

The plan of this paper is as follows: Section I contains a description of 
permutation decoding in general, without reference to the particular 
alphabet we wish to decode. Section II is an example; it contains a de- 
tailed account of a particular permutation group which will suffice to 
decode many binary cyclic alphabets. Section III describes how the prob- 
ability of improper correction and of detection without correction may 
be estimated. 


I, ERROR CONTROL AND PERMUTATION DECODING 


Let V” be the set of all possible binary sequences of length n.* The 
distance between two sequences is the number of places in which they 


* The method will work equally well for multilevel codes. All that is needed is 
to find a euphonious substitute for the term ‘‘binary sequence.’”’ 
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differ; the distance between 2; and v, is the minimum number of bits we 
must change in 1 in order to convert it into ve. 

For purposes of error control, some sequences of V” are designated as 
the sequences which will be transmitted. This subset of V” is called a 
code. Error detection consists in finding out whether a received sequence 
belongs to the code. Every method of error correction consists in map- 
ping a received sequence onto the nearest member of the code, where 
nearness is defined in terms of the distance function defined above. If 
there are several nearest members, the correction procedure chooses one 
in some arbitrary fashion or indicates that an uncorrectable error has 
been found. 

The strategy for choosing a code is usually to place its members as far 
apart as possible in V”. It will then take a relatively large number of 
errors to cause a transmitted code sequence to be received as a different 
code sequence. If the distance between any two code sequences is 
=2e + 1, it is theoretically possible to correct all single, double, --- 
e-fold errors. This may be restated as follows: If v is a received sequence 
in which Se errors have occurred, there is a unique code sequence a at 
distance Se from v. Every other member of the code is at distance 
>e from »v. 

The business of decoding is to find a, given v. To do this expeditiously 
we need some additional structure in the code, and from now on we re- 
strict our choice of codes to the kind described in the next paragraph. 

An alphabet™ (systematic code, group code) is one in which a fixed 
number of fixed bit positions are designated as information places, and 
the other bit positions contain parity checks, which are linear combina- 
tions of the contents of the information places. For convenience, the first 
k bit positions are taken to be the information places. From any k-place 
binary sequence h we obtain a unique letter of the alphabet by adding 
n — k parity checks. This letter will be denoted by m(h). 

Let & stand for the first k coordinates of the n-place sequence a. a is 
a letter of the alphabet if and only if a = m(a@). Let 7 be a permutation 
of bit positions in V” which preserves the alphabet; if a is a letter, so is 
am. The first k positions of am are information places, and ar = m(a7), 

Let v be a received sequence containing Se errors. If no errors have 
occurred in the first & places of v, ao = m(#) is the unique letter of the 
alphabet at distance Se from v, and is the corrected version of v. On the 
other hand, if one or more errors have occurred in the first k places of v, 
the letter m(#) is not the corrected version of v, since it is the same as v 
in the first & places. In this case m(%) is at distance >e from v. 


* It has been shown! that every systematic code is a group code and that every 
group code is a systematic code. 
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The first step in the decoding procedure is to form ay = m(%), find the 
distance between v and a», and take ap as the corrected version of v if 
this distance is Se. 

Let a denote the unique letter of the alphabet at distance Se from v. 
Let 7 be, as before, a permutation of bit positions which preserves the 
alphabet. Clearly the distance between am and vz is the same as that 
between a and »v. 

Suppose that we can find a permutation 7; which preserves the alpha- 
bet and which moves the errors in v out of the first k positions. Then 
a; = m(v7;) is at distance Se from vz; , and is the unique letter of the 
alphabet with this property. Consequently a = a;r; is the corrected 
version of v. 

This suggests the following decoding procedure: Let J (the identity), 
71,72, °°* be a sequence of permutations which preserve the alphabet. 
Form the letters 


ao = m(vl), a= m(vm), a = m(vm), °° 


and at each step find the distance between a; and vm; . Continue until a 
letter a; is found which is at distance <e from vm;. Then a;r;_ is the 
corrected version of v. 

A received vector which is at distance >e from all letters of the alpha- 
bet will be detected as an error but not corrected by this procedure. Some 
provision must be made for this eventuality. This is discussed in Section 
Til. 

It is also possible for the decoder to make an incorrect “‘correction.”’ 
This will happen if an error pattern (of more than e errors) causes the 
transmitted letter a to be received as a sequence v which is at distance 
<e from a different letter a’. The probability of this occurrence is cal- 
culated in Section ITI. 

In order for permutation decoding to work, we must be sure that one 
of the sequences vz; is correct in the first & places. If v = a + f, f being 
the error sequence, the permutation 7; must move all nonzero coordi- 
nates of f out of the first k places. In order for the procedure to be prac- 
tical, it must be possible to move all sets of Se errors out of the first k 
places with a fairly short sequence of permutations. 

To correct all sets of Se errors in a block of length n, we need the fol- 
lowing: (2) an alphabet of block length n, dimension k, and minimum 
distance 22e + 1; and (77) a set of permutations, m , m2, --- which 
preserve the alphabet and at the same time move any set of Se errors 
out of the first k places. 
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We emphasize that the reason for insisting that the permutations 7; 
shall preserve the alphabet is to keep the parity check calculation always 
the same. The encoder (the parity check calculator) is a complicated and 
expensive piece of equipment; it is desirable to use only one encoder in 
the decoding scheme. If for any reason (such as real time operation) it is 
necessary to have more than one encoder, we can, to a certain extent, 
relax the restriction on the permutations 7; . 

It may seem to be quite a trick to find at the same time both a suit- 
able alphabet and a suitable set of permutations; really the chief diffi- 
culty is that neither alphabets nor permutation groups have been studied 
from this point of view. It is shown in Section II that a very simple per- 
mutation group will do for many cyclic alphabets. 

We conclude this section with an example of permutation decoding 
applied to the Hamming alphabet with n = 7,k = 4, ¢ = 1.* The alpha- 
bet is written out in Table I; it is seen to be invariant under cyclic per- 
mutation. 


TaBLe I—A Cycric ALPHABET WITH n = 7,k = 4,e = 1 


00000 0 0 
01101 0 0 
0011010 
0600110 1 
1000110 
0100011 
10100041 
1103100 0 
ltt A ae bs 
1603104141 
1100431041 
I 110041 0 
011410041 
101314100 
01014141 é=+0 
001014121 


Let 7 denote the cyclic permutation. Clearly at most four applications 
of T will move any single error out of the first four places. The decoding 
sequence consists of the permutations J, 7, T’, T°, T*.t 

Let the received vector be 1110100 (the first nonzero vector of Table 
I with an error in the first place). The successive stages of the decoding 
process are shown in Table IT. 

* The example is chosen for simplicity. Permutation decoding is not the most 
efficient way of correcting single errors. 


} E. R. Berlekamp has pointed out that the shorter decoding sequence J, T3, 
T® is sufficient. 
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TABLE II — DecoDING PROCEDURE FOR THE ALPHABET OF TABLE I] 


V = 1110100 
m(V) = 1110010 distance = 2 
VT = 0111010 
m(VT) = 0111001 distance = 2 
VT? = 0011101 
m(VT?) = 0011010 distance = 3 
_VT? = 1001110 
m(VT?) = 1001011 distance = 2 
VT* = 0100111 
m(VT*) = 0100011 distance = 1 


Thus a = 0100011 is the unique letter at distance <1 from V7, and the cor- 
rected version of V isaT~4 = a7 = 0110100. 


II. PERMUTATION DECODING OF CYCLIC ALPHABETS* 


The coordinate places in V” are labeled by the numbers 0, 1, 2, --- , 
n — 1. This notation is convenient for describing permutations. If w 
stands for one of these numbers, the cyclic permutation is 


T:a—w + 1 (addition mod n). 
The powers of the cyclic permutation are 
T:w—3w + 2; T saa + Bye, Pg sg An: Say 


A cyclic alphabet in V" is an alphabet which is invariant under T, 
hence also invariant under T”, 7”, etc. We assume that we wish to decode 
a cyclic alphabet with parameters n,k,e. 

Successive cyclic shifts will eventually bring any k consecutive bits to 
the first k positions, and hence will move out of the first k positions any 
error pattern in which there is a gap of length =k. In particular, the se- 
quence I, 7, --- , T” ’ will always correct all single errors. + 

This sequence will not correct an error pattern in which there is no 
gap of length =k. Suppose, for example, that n = 23, k = 12. The error 
pattern shown below cannot be corrected by cyclic shifts alone. 


X12345678 X 10 11 12 13 14 15 16 17 18 X 20 21 22 


To deal with such cases we introduce another permutation U: w — 2w 
(multiplication mod 7), and its powers, U*: w > 4w, U*: w — 8w, ete. 
If n is odd, there exists a least integer ¢ such that 2° = 1 mod n; and 
U' = I. The choice of U is motivated by the following theorem. 

* It is to be emphasized that this section is only an example. The permutations 
described here will not suffice to decode all cyclic alphabets of odd block length. A 


method of finding other permutations is given in Appendix A. 
7 It will also correct all double errors if k < n/2, and so on. 
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Theorem 1: Every binary cyclic alphabet of odd block lengthis invariant 
under U and the powers of U. 

The proof of this theorem is given in Appendix A. 

The error pattern 0, 9, 19 above is changed by U into 0, 18, 15. This 
pattern is moved out of the first twelve places by 21 cyclic shifts. 

The permutation group on 0, 1, --- ,n — 1 generated by 7 and U will 
be called G,, . It is easy to check that TU = UT"; hence we may repre- 
sent every permutation in G, in the form U‘T’, withO < i < ¢ — 1, 
0 Sj Sn — 1. Nowevery power of U leaves 0 fixed, and no power of 
T (except the identity) leaves 0 fixed; thus U’T’? = U"" if and only if 
t = hmodtandj = k mod n. It follows that the group G, is of order 
nt and consists of the permutations: 


t, z, Tr are | ie 

U, UT, UT’, ++, ur 

U’, UT, Oi kee vee U7" 

yt yep riage : taped 
Let 0 S um <u < ++: < us S n — 1 (n odd), bea set of integers; 
we suppose that errors occur in places % , --- , us. Let g(w, +--+ , Us, 2) 


be the length of the maximum gap which can be inserted in this sequence 
by repeated multiplication by 2 mod n. 

If u,, denotes that integer less than n which is congruent to 2”u, mod n, 
then 


g(t, °++,Us,n) +1 = Max | wis — use I, 
Udy 


under the condition that the interval [wu , uj.) contain no other u,; . 
Let g(s,v) be the minimum value of this maximum gap for all possible 
choices of the s values wu, +--+ , Us. 


g(s,n) = Min g(u 9 7°° Us, n). 
Uys tery 


The group G, then contains a permutation which moves any set of s 
errors out of the first g(s,n.) places. Clearly s’ < s implies g(s’,n) = g(s,n). 
Hence a binary cyclic n,k,e alphabet with n odd may be decoded by Gn 
if and only if k Sg(e,n). The quantity g(e,n) has a few obvious prop- 
erties. 

The numbers 0, 1, --- ,2 — 1 can be partitioned into subsets which are 
invariant under U.* For example, for n = 15, these subsets are 

* The number of cyclic alphabets of block length n is determined by the num- 


ber of these subsets; the dimensions of the cyclic alphabets are determined by the 
sizes of the invariant subsets; see, for example, Ref. 4. 
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(0), (1,2,4,8), (3,6,12,9), (5,10), (7,14,13,11). 


The union of any number of invariant subsets is also invariant under 
U; from these subsets we may obtain upper bounds on g(e,n) by in- 
spection. In the example above we obtain: 

g(2,15) S 9, since the invariant set (5,10) gives us a maximum gap 
11,12,13,14,0,1,2,3,4 of length 9. 

ae S 4, since the invariant set (0,5,10) gives us a maximum gap 

2,3,4 of length 4. 

ae 15) S 2, since the invariant set (0, 3,6,9,12) gives us a maximum 
gap of length 2. 

These upper bounds limit the usefulness of the group G, . However 
it is still sufficiently useful to be of interest. 

The value of g(e,n) for various choices of en have been computed 
on the IBM 7090. These are tabulated in Table III together with the 
parameters n,k,e for several cyclic alphabets. 


TasBLe II] — Errect or PermMutTATION U For 
DIFFERENT Buiock LENGTHS 


Code Gap Length 
n k e g(2) —-g(3)_~—g (4) (5) 
47 24 5 26 
31 21 2 25 
31 16 3 19 
31 lI 4 12 
23 12 3 17 
21 12 2 13 
21 9 38 6 
21 5 4 6 
17 9 2 13 
15 7 2 9 
15 5 3 4 








The gap length g(s) is the maximum number of consecutive error-free positions 
which can be inserted into an arbitrary pattern of s errors in n places by succes- 
sive applications of the permutation w — 2w mod n. If k S g(e), the group gener- 
ated by 7’ and U contains a sequence of permutations which will suffice to decode 
an n, k, e alphabet. 

It is desirable, if possible, to use only part of G, in the decoding se- 
quence. As an example we consider the alphabet with n = 23, k = 12, 

= 3.” In this case one of the permutations, U, U’, U” will always 
create a gap of length at least 12 in any set of 3 errors. The decoding 
sequence is:T 

* This alphabet is described in detail in Table V. 


+ It has been shown by E. R. Berlekamp that a subsequence of length 40 is all 
that is really necessary. 


wo) 
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i, T,, Y Ree T” 


U.. U8... UP. ese UT? 
U7’, UT, UT", wee ie pe 
(te UST. UE: wes, OG saad ie 


The decoding procedure for this particular alphabet has been simulated 
on the IBM 7090. A diagram of the logical program is given in Fig. 1, 
and Table IV traces a particular sequence through the decoder. In prac- 
tice it is convenient to add one more permutation (in this case T'U) to the 
decoding permutations, so that a sequence passing through the entire 


INPUT =V 


=n) 


It 
R 


W = WU? i=o W= WT, L=L+1 W=a 
i =a je >| 
a=W) 


W = WU OUTPUT =W 


Fig. 1 — Logic of decoder for (23,12,3) alphabet. The contents of the boxes are 
Hvar Reg instructions; for example, 7 = 7 + | is to be interpreted as ‘‘replace 
ibyt+1.” 


TABLE IV — DEecopING PROCEDURE FOR THE (23,12,3) ALPHABET 


v 3.6 7 iil 
oT 04 6 8 
pT 22 2 4 6 10 
vU 383.5 6 7 
vUT 04 6 7 
vUT2 24 5 6 
vU? 3 5 6 7 
v7 46 7 8 
ve2T 2 01 38 4 
af?TLR 01 3 4 
al?2T2 25 6 9 
a0 0 9 10 Ii 
ap 0 1 10 Ii 
aGYi0722 8 9 10 12 
a 0 3 #5 £ «Ii 


The numbers indicate the positions of the ones in the 23-bit sequence. For 


00010101000101110011111. 


13 
12 
12 
10 

8 

9 
10 
Il 

8 

8 
12 
13 
12 
13 
13 


ORPRWNOWNOW 


ROD OUR BOTH HB Ort 


WTO ODD HOT 


— rs 
ONTO NATO COM 


13 
16 


10 


10 
8 


Parity Check 


Distance 


12 138 14 20 21 22 
14 17 18 19 21 22 
12 13 19 20 21 22 


Woh OD ONIOn oO 


example, the first sequence v is 


P6P 
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set emerges in its original form. The final output of the permuting regis- 
ter is then the corrected form of the received sequence. 

The operation of the 7090 program is of course sequential—it employs 
one subroutine to simulate the parity check calculation. It is clear from 
the logic diagram that it is quite convenient to split the encoder into four 
parallel sections, each of which contains a register capable of making a 
cyclic permutation and an encoder to calculate parity checks. This idea 
can be applied to speed up the decoding of any cyclic alphabet. 


WI. EVALUATION OF PERMUTATION DECODING AS A MEANS OF ERROR 
CONTROL 


Permutation decoding of an n,k,e alphabet @ will map a received se- 
quence v onto the nearest letter of @ provided that this letter is unique. 
This is the case if v lies at distance Se from some letter of @. If v is at 
distance >e from every letter of @ the decoder will detect an error but 
will be unable to correct it.” 

The decoder will also make mistakes. If f is an error sequence of more 
than e errors, and a the transmitted letter, the received sequence a + f 
may lie at distance Se from some other letter a’ of @. The decoder will 
then interpret a + f as a’. The error sequences which cause such in- 
correct decoding are characterized by the following theorem. 

Theorem 2: The error sequences which cause the decoder to ‘“‘correct’’ 
incorrectly are exactly those sequences of weight >e which le at distance 
Se from some letter of @. 

Proof: Let f be a sequence of weight >e such that f = 8B + f’,B€@, 
f’ of weight Se. 

For any transmitted letter a 


atf=atBpt+f', 


and the decoder will interpret a + fasa + 8. 
Conversely, suppose that f is an error sequence such that a + f is 
decoded as a’ + a. Then 


atf=a4+f' a’ ¢ G,f’ of weight Se, 
and 
f=a—-atf’. 
Hence f is at distance Se from the letter a’ — a of @. 
* One is tempted to suggest that the decoder ask for retransmission of such de- 
tected errors. This idea is of dubious value; error correction by decoding should 
not be used at all if error correction by detection and retransmission is a possible 


alternative. We must assume that retransmission is extremely awkward, if not 
completely infeasible. 
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Let A(s),s = 0,1, ---, be the number of letters of @ of weight s. Let 
C(s), s = 0, 1, ++, ~ be the number of sequences of V” of weight s 
which lie at distance Se from letters of @. The C(s) are uniquely de- 
termined by the A(s), and may be obtained from them by a simple 
calculation; the exact formula is given in Appendix B. The values of 
A(s) for a number of binary cyclic alphabets are tabulated in Table V, 
and the values of C'(s) for these alphabets are given in Table VI. 


Fors S$ e, C(s) = (") and is the number of sequences of weight s 


which are properly corrected by the decoder. For s > e, C(s) is the 
number of sequences of weight s which are improperly corrected by the 
decoder. 

Let D(s),s = e+ 1, ---,n be the number of error sequences of weight 
s which cause the decoder to detect an error that it cannot correct. 


Clearly D(s) = Ofors S e, and D(s) = (") — C(s) fors > e. 


P, denotes the probability that a received sequence will be “‘corrected”’ 
incorrectly by the decoder; Pp denotes the probability that a received 
sequence will be detected as an error but not corrected. We consider 
first a binary symmetric memoryless channel with bit error probability 
p. The probability of s specific errors in a block of length n is then 
p (i — p)””*, and this probability is independent of the location of 
the errors. Hence 


n 


C(s)p*(1 — pp)", 
1 


n 


D(s)p*(1 — p)”™. 
+1 


P,(BS.) 


Il 


Sue: 


P,(BS.) 


s=—e 


It is to be noted that if error correction by detection and retransmis- 
sion is used, the probability of an undetected error is 


>> A(s)p'(1 — p)”>; d=2e+1 or 2e+ 2. 
s=d 


This sum starts with the first nonzero value of A(s) (for s > 0), ie. 
with s = 2e + lors = 2e + 2. It is obvious that for the values of p,n 
currently in use in the Bell System, error correction by detection and 
retransmission is the preferable scheme. 

The values of Pz(B.S8.) and P»(B.S.) for a number of alphabets are 
tabulated in Table VII; p is taken to be 3.22 X 10~°, the over-all bit 
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error rate on the telephone network obtained from the Alexander, Gryb 
and Nast task force data.” 

Except for the first and last example, the alphabets in Tables V, VI 
and VII occur in pairs. The second alphabet in each pair consists of 
the letters of even weight in the first. Since the minimum distance of 
the second alphabet is 2e + 2, its value of C(e + 1) is zero. In other 
words, every error of weight e + 1 will be detected. If this is important, 
it is advantageous to use the second alphabet. 

The error rates of Table VI are fantastically low; unfortunately the 
fantasy resides in the binary symmetric channel. The situation is very 
different on the real telephone channel. 

Let P(s,n) be the probability of s errors in n consecutive bits. [For 


the binary symmetric channel P(s,n) = (") o (1 — p)” *.| Tables of 


P(s,n) for the telephone network have been calculated from the Alex- 
ander, Gryb and Nast task force data. 

The decoder will either detect or correct wrongly every error of weight 
>e. Hence 


n 


Py+ Pp = » P(s,n). 


s=e 


It is impossible to obtain exact formulae for Pz and Pp separately. 
Using the methods of Ref. 5 we obtain the approximate formulae 


P,(T.N.) = > C(s) Piet) 
(:) 
PATNY = >> Diy. 


s=e-+1 nr 
s§ 


These numbers have been computed and are tabulated in Table VIII. 
This shows rather clearly that on the channel described by the Alex- 
ander, Gryb and Nast data the word error rate with error correction 
is greater than the bit error rate with no encoding. 

Of course the average error rates of Table VIII conceal something. 
Examination of the P(s,n) tables for the individual calls shows that 
about half of the calls in which errors occurred would be handled suc- 
cessfully by permutation decoding. 


47,24,5 
A(s)/47 


1/47 


TABLE V — VALUES oF A(s) 


31,21,2 
A(s)/31 
1/31 
7 

27 


75 
245 


655. 


1387 
2640 
4480 
6510 
8310 
9489 
9489 
8310 
6510 
4480 
2640 
1387 
655 
245 
75 
27 

7 
1/31 


31,20,2 
A(s)/31 
1/31 

0 
27 


0 
245 


0 
1387 
0 
4480 
8310 
9489 
6510 
0 
2640 


31,16,3 

A(s)/31 
1/31 
5 


15 
168 
280 
589 
589 
280 
168 

15 


5 
1/31 


31,15,3 
A(s)/31 
1/31 

0 


15 
0 
280 
0 
589 
0 
168 
0 

5 

0 


867 
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23,12,3 23,11,3 (21,12,2) (21,11,2) (17,9,2) (17,8,2) (15,7,2) 


A(s)/23 A(s)/23 s A(s) A(s) 5 A(s) A(s) s A(s) 
1/23 1/23 0 ] 1 0 1 1 0 1 
11 5 21 0 5 34 0 5 18 
22 22 6 168 168 6 68 68 6 30 
56 0 7 360 0 7 68 0 7 150 
56 56 8 210 210 8 85 85 8 150 
22 0 9 280 0 9 85 0 9 30 
11 11 10 1008 1008 10 68 68 - 10 18 
1/23 0 ll 1008 0 11 68 0 15 1 
12 280 280 12 | 34 34 
13 210 0 17 1 0 
14 360 360 
15 168 0 
16 21 21 
21 1 0 


DNIGOOAd NOILVIAWYad 


667 


TABLE VI —VAa.uuEs or C(s) 





n,k,e = 47,24,5* 31,21,2* 31,20,2 31,16,3 31,15,3 
s=1 C(s) = 47 31 31 31 31 
2 1081 465 465 465 465 
3 16215 2170 0 4495 4495 
4 178365 138640 12555 5425 0 
5 1533939 82274 5022 29295 26040 
6 1997688 360964 339047 92225 18020 
7 11700743 1276115 81685 329375 303180 
8 58503719 3829585 3591040 1248525 86025 
9 253516120 9788250 604655 3190675 2861455 
10 1094459500 21506932 20159981 6790333 690525 
11 3681363800 41087771 2569497 12963363 11812395 
12 10764415000 68535730 64275400 21284445 1987720 
13 28981118000 100106900 6245260 31108034 28201320 
14 70307802000 128661310 120616350 40561485 3675360 
15 154677160000 | 145890120 9085914 45969682 41569263 
16 310193350000 145890120 136804210 45969682 4400419 
17 565646700000 128661310 8044965 40561485 36886124 
18 942103590000 | 100106900 93861645 31108034 2906715 
19 1487947500000 68535730 4260330 21284445 19296724 
20 2012287600000 41087771 388518275 12963363 1150968 
21 2587226900000 21506932 1346950 6790333 6099808 
22 3059047600000 9788250 9183595 3190675 329220 
23 3325051800000 3829585 238545 1248525 1162500 
24 1276115 1194430 329375 26195 
25 360964 21917 92225 79205 
26 82274 77252 29295 33255 
27 -+- symmetric 13640 1085 5425 5425 
28 terms 2170 2170 4495 0 
29 465 0 465 0 
30 31 0 31 0 
31 1 0 1 0 
n,kye = 23,12,3 23,11,3 21,12,2 21,11,2 17,9,2 17,8,2 15,7,2 
s=1 (|C(s) = 23 23 21 21 17 17 15 
2 253 253 210 210 136 136 105 
3 1771 1771 210 0 340 180 
4 8855 0 2625 2520 1190 1020 540 
5 33649 28336 10269 1008 3910 408 1413 
6 100947 14168 24024 21168 7820 6936 2355 
7 245157 216568 52440 4200 11560 1428 3135 
8 490314 61226 92610 85050 14450 13005 31385 
9 817190 715990 1315380 12810 14450 1445 2355 
10 1144066 138188 161196 146748 11560 10132 1413 
l1 1352078 | 1180774 | 161196 14448 7820 884 540 
12 1352078 171304 1381530 118720 3910 3502 180 
13 1144066 | 1005928 92610 7560 1190 170 105 
14 817190 101200 52440 48240 340 340 15 
15 490314 429088 24024 2856 136 0 1 
16 245157 28589 10269 9261 i7 0 
17 100947 86779 2625 105 1 0 
18 33649 5313 210 210 
19 8855 8855 210 0 
20 1771 0 21 0 
21 253 0 1 
22 23 0 
23 1 0 





* This table is correct to eight significant figures. 
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TaBLE VII — Error RAtTEs FoR THE BINARY SYMMETRIC CHANNEL 


n,k,e 





47 24,5 


31,21,2 
31,20,2 


31,16,3 
31,15,3 


23,12,3 
23,11,3 


Pr 


2.23 X 


7.23 X 
1.342 X 


5.83 
9.01 


9.59 
9.81 


7.00 


XxX XX KX KX 


me bo 
mor ow 
ek Oo 


6.092 X 


10-21 


10-11 
10722 


10715 
10719 


190715 
10719 


10712 
1Q715 


10-2 
10718 


10-12 


Pp 





4.56 


0.77 
1.50 


2.80 
3.38 
9.59 


3.72 
4.42 


1.80 
2.27 


1.078 X 


xX XK XS KK KX X 





10720 


190710 
190710 


10714 
10-4 
190715 


190711 
10-4 


1Q711 
19g711 


10710 


Pi+ Pp 
4.738 X 107*° 
1.50 X 1071° 
1.50 X 10718 
3.38 X 107 
3.38 XX 1074 
9.59 xX 10715* 
9.59 X 10715 
4.42 X 1071 
4.42 X 107 
2.27 X 1071! 
2.27 X 1074 
1.078 X 1071° 


* This is a close-packed alphabet; every sequence of 23 binary bits is at dis- 


tance $3 from some letter of the alphabet. 


CONCLUSION 


Permutation decoding is a simple and feasible scheme for error correc- 
tion without retransmission. It is particularly suitable for use with a 
highly redundant alphabet. Like any such scheme it produces many more 
undetected errors than error correction by detection and retransmission, 


Taste VIII — Error Rares ror THE TELEPHONE NETWORK* 


n,k,e 


47 ,24,5 


31,21,2 
31,20,2 


31,16,3 
31,15,3 


23,123 
93,11,3 


* Py and Pp are approximate values. Pz + Pp is exact. 
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but it is quite adequate for a channel in which P(s,n) decreases rapidly 
as s increases. It is of very doubtful value on the telephone network as 
described by the Alexander, Gryb and Nast data. 
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APPENDIX A 


Idempotents and Automorphisms of Cyclic Codes 


We give first a short summary of the properties of cyclic alphabets. 

Let V be a finite field of characteristic g. Let V” denote the direct sum 
of n copies of V. 

Denote by V[y] the ring of polynomials in y over the field V. Let 
V(x] = Viyl/(y” — 1) be the residue class ring of V[y] mod y” — 1. 
V[z] consists of all polynomials of degree <n — 1 with coefficients in V. 
Addition of polynomials is done as usual; to multiply two polynomials 
we multiply in the usual way and then reduce exponents mod n. 

A subset @ of polynomials of V[z] is called an ideal if 


(4) 91,92.€A> agi + age € GQ, a1,a,€V 
(21) geQ>19 €@. 


A polynomial is completely determined by its coefficients; it is possi- 
ble, in fact, to identify V[z] and V”. However, it is convenient to regard 
them as separate entities, related by the (1-1) mapping 


—1 
ao + ae -+::- + anv" 2 a0, 1, "ety Oni. 


An ideal in V[z] is, by property (2), a linear subspace of V”. By 
property (77) it is invariant under a cyclic permutation of coordinates; 
hence it is a cyclic alphabet in V”. Conversely, a cyclic alphabet in V” 
is an ideal in V[z]. We represent the ideal and the alphabet by the same 
letter, @. 

The ring V[z] may be regarded as the group algebra of the cyclic 
group 1, z, --- , 2” over V. The group algebra is semi-simple provided 
that q does not divide n (Ref. 6, Section 10.8). In this case it is known 


that every ideal contains a polynomial e with the following properties 
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(4) eis idempotent. (ce =e) 
(it) eis a unit for @. (aeQ= ae = a) 
(zit) e generates @. (@ consists of all polynomials 
fe, f ¢ V[x)). 


e will be called the generating idempotent of @. 

An automorphism o of V[a] is a (1-1) mapping of V[z] onto itself 
which respects both addition and multiplication. If v , ve « V[«], then 
(vy + V2)o = Vio + Voot 

(v2) = (140) (vec). 
Lemma 1.1: An automorphism o of Vix] preserves an ideal @ if and 
only af o preserves the generating idempotent e of @. 
Proof: Suppose that o preserves e. Then @o = V[x]-eo = V[z]-e = @. 
Suppose that o preserves @. Then ec ¢ @, and ese = eo by property 
(12). . 
Let b be the element of @ such that bo = e. Then 


ece = eobc = (eb)o = bo = @. 


Hence e = ece = ea, and o preserves e. 

Lemma 1.2: If q (the characteristic of V) ts relatively prime to n (the 
block length of @), then the mapping o: x* — x" is an automorphism of 
V (a). 

Proof: Clearly this mapping respects addition and multiplication in 
V(x]. We have only to show that it is 1-1. 

If wt = «”, then (¢ — j)qg =0 mod n. Since q is prime to n, this 
implies that 7 — 7 =0 mod n or2* = x’. 

This proves the lemma. 

Theorem 1.3: If q ts prime to n, every ideal in Vx] ts preserved by the 
mapping o: x' > 2", 


Proof: Let @ be an ideal in V[z] ande = >> ax’, the generating idem- 
t=0 
potent of @. 
c6 = Yavt= (= aat* 0", 
7=0 i=0 


since q is the characteristic of V. 

e=¢ =e =--- = e;hencec preserves e, and by Lemmas 1.1 and 
1.2, 0 preserves @. 

Let the coordinate places of V” be labeled 0, 1, --- , m — 1; the map- 
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ping ¢: x’ > x in V[z] corresponds to the permutation U,: w > qu of 
coordinate places in V”. 

Corollary: Hvery binary cyclic alphabet of odd block length is preserved 
by the permutation U: w > 2w. 

This is Theorem 1 of Section II. The proof given here contains more 
machinery than is necessary to prove Theorem 1. This is done on purpose, 
in order to get Lemma 1.1, which suggests a method of finding other auto- 
morphisms of V[«] which preserve a particular cyclic alphabet. 


APPENDIX B 


Distribution of Weights in the Cosels of a Group Code 


Let @ be an (n,k,e) binary alphabet, and let @ be the orthogonal 
complement (dual alphabet) of @ in V”. Let A(z), B(z) denote the 
number of letters of weight 7 in @, ® respectively. The quantities A (7), 
B(i) are connected by the generating function.” 


SAC) + 2)" = 2) = 2 D BUDe! 


Since the A(z) are known, the B(7) may be calculated from this rela- 
tionship. 
Set 


ip G-g = Lvaae 


Let C(s,j) denote the number of sequences of weight s in V” which 
are at distance 7 from some letter of @. Then if7 S e, C(s,7) and B(t) 
are related by the generating function.’ 


Y BOWE + 2)U = 2) = 2" Y Clas)s" 


Since B(z) and (2,7) are known, C'(s,j7) may be calculated from this 
relation. 
Clearly 


O(s) = » C(s,f). 
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Optical Maser Oscillators and Noise 


By EUGENE I. GORDON 
(Manuscript received September 5, 1963) 


The transmission line matrix formalism so useful for describing the 
transfer properties of microwave networks 1s extended to the electromagnetic 
fields associated with optical masers. The spontaneous emission noise of 
the optical maser 1s examined and shown to be amenable to a thermal de- 
scription. Taking the point of view, well accepted at microwave frequencies, 
that a weakly nonlinear oscillator is a saturated amplifier of noise, the 
power and linewidth of the noise radiation emitted by the optical maser 1s 
calculated using the transmission line formalism. The significant param- 
eters for any optical maser are shown to be the frequency, the single-pass 
gain of the maser medium, the effective mirror reflectivity and the population 
ratto. The pre-oscillation characteristics of the maser are examined and 
the reason for the extremely sharp oscillation threshold of the gas masers 
ts discussed. Some observations concerning semiconductor optical masers 
are also made. 


I. INTRODUCTION 


This paper represents an attempt to describe the optical maser or 
laser from a microwave circuit point of view and is largely tutorial, 
since many of the results obtained froma circuit viewpoint are already 
known. The generality of the method of approach enlarges their area of 
validity, however. 

Many of the people working on optical masers who do not have a 
background in microwave theory and techniques may find a fresh point 
of view. In particular, they may find a very modest introduction to an 
extremely well developed store of computational techniques which are 
applicable to optical masers. This may save them the trouble of in- 
venting their own. 

On the other hand, those who have previously been working in the 
field of microwaves may find that the analogies between optical masers 
and more conventional microwave devices are more cogent than they 
had appreciated. Finally, it is hoped that some of the distinctions be- 
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tween oscillating and pre-oscillating or subthreshold masers will be 
clarified. 


II. THE CONVENTIONAL OSCILLATOR 


Excluding strongly nonlinear oscillators with periodic but non- 
sinusoidal waveforms, it is often stated that an oscillator is a device 
having an internal gain which exceeds its total losses. Supposedly, 
noise triggers it off and it then continues to put out oscillatory power at 
a level determined only by saturation effects. The steady-state satura- 
tion level is defined as that for which the internal gain just equals the 
loss. An extensive discussion of microwave oscillators, based on this 
point of view, is given by Slater. 

Although this point of view often constitutes a good working defini- 
tion of a feedback oscillator, it is incomplete in that it neglects the 
continuing presence of the noise. As a result, when the internal gain of 
the oscillator exactly equals the losses, so that the effective lifetime of a 
photon in the feedback loop is infinite, the noise power output of the 
oscillator must increase without limit. Similarly, the associated line- 
width of the noise output must be zero. Since this situation is physically 
unrealizable, it is clear that the noise must be taken into account and 
that the steady-state gain could never exactly equal the loss and must 
always saturate at a slightly lower value. As a result, there could be no 
continuing, self-sustained oscillation which starts from noise. 

From these considerations, it appears that a better, but still incom- 
plete, description of an oscillator would be to say that a steady-state 
regenerative oscillator is a feedback amplifier driven to saturation by 
a noise input. Internally produced noise is usually the driving force; 
however, an additional source may be the noise entering through the 
output port. The external gain of the amplifier, that is, the gain ex- 
perienced by any input signal, is usually extremely large unless the 
amplifier is saturated by the input signal. Since the amplification is 
obtained regeneratively, that is, by the use of feedback, the bandwidth 
of the gain is limited; the higher the gain the more limited it is. The 
amplified, narrow-band noise output is the output signal of the oscil- 
lator. When the large gain can be obtained without regeneration, the 
noise output need not be narrow-band. 

This concept of the oscillator as a saturated amplifier of noise is not 
new and is well known in the microwave art. More recently, this concept 
has been employed by Gordon, Zeiger and Townes? in their treatment 
of the microwave maser oscillator, and by Wagner and Birnbaum,’ 
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Schawlow and Townes,’ Shimoda,® Blaquier® and Fleck’ in their treat- 
ments of the optical maser oscillator. 

While this definition of an oscillator is somewhat more satisfying, it 
implies that an oscillator is merely an extremely narrow-band filter 
with gain. As a result, the statistical properties of the noise input should 
be preserved, except for the spectral narrowing. For example, a filtered 
Gaussian noise input® would remain Gaussian. Since a narrow-band 
Gaussian noise process shows amplitude fluctuations with a time con- 
stant approximately the inverse of its spectral range,’ the output of a 
filter with gain should exhibit this property also. The fact that a true 
oscillator does not indicates that a correlating mechanism is operative. 

Gain saturation is one mechanism that operates to eliminate fluctua- 
tions in the output intensity. The action is similar to that of a limiter. 
In an optical maser, gain saturation arises almost entirely from deple- 
tion of the population inversion rather than from any nonlinearity in 
the stimulated emission process. To a very high degree, the output wave- 
form is sinusoidal and the saturation depends only upon the time- 
average power, the time average extending over a time long compared 
to the period of the output waveform but short compared to any relaxa- 
tion mechanism or pumping rate. 

Suppose now that the filtered output has a spectral range Av so that 
the input noise has power fluctuations with a time constant approxi- 
mately Av. If the gain of the maser medium has a relaxation time 
t, < Ay”, then the input fluctuations will be virtually absent in the 
output. On the other hand, if 7, > Av” the input fluctuations will 
appear in the output. 

Thus, for a true oscillator 


tgAv <1 (la) 
while for an amplifying filter 
T,Av > 1. (1b) 


As will be seen later, for a weak optical maser Av can be quite large, 
while for a strong maser Av becomes vanishingly small. The cavity 
bandwidth Ay, represents an upper limit for Av. For example, in a gas 
maser Ay, & 10° cps and 7, % 10°, so that for unsaturated gains less 
than the loss, Av } Av, , Avr, © 1, and the device acts like an amplifying 
filter. When the unsaturated gain exceeds the loss, Avy becomes much 
less than Ay, and the device becomes an oscillator. 

In the semiconductor maser, the lowest reported value is Av 3 X 10° 
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eps, corresponding to 0.1A. Unless 7, < 3 X 10”, it is quite probable 
that the device acts like an amplifying filter rather than a true oscillator. 

The interested reader will find a short discussion of the effect of noise 
on oscillators in a book by van der Ziel.? His discussion indicates a 
method of approach to obtaining a quantitative solution to a very 
complicated nonlinear problem. However, the concept of the amplifying 
filter probably yields a good first approximation to the spectral width 
of the oscillator output. 

In the following sections, the foregoing concepts will be exploited to 
exhibit the circuit formalism and to study linewidth and threshold be- 
havior. The basic results are applicable to any uniformly pumped, single- 
mode maser or any multimode maser for which nonlinear mixing or 
coupling of modes is not significant. Some remarks concerning the lack 
of extremely narrow linewidths in the semiconductor maser will also 
be made. 


III. SINGLE MODE REPRESENTATION FOR AN OPTICAL MASER 


The electromagnetic fields associated with the optical maser are very 
close to being plane waves. To a very good approximation, each mode 
of the electromagnetic field can be represented by field quantities, 
E(z) and I(z). These quantities can be normalized to have the dimen- 
sions of voltage and current, respectively. The relationship between 
E(z) and I(z) is obtained by specifying the value of the function Z(z) = 
E(z)/I(z) at some point z. In the content of this paper, a mode is one 
member of a complete set of transverse eigenfunctions which are appro- 
priate for the geometry in question. No orthogonality with respect to 
the z coordinate is implied. 

If the various modes of the electromagnetic field are uncoupled and 
E and I are represented as complex quantities, then a linear relationship 
of the form 














E(a)|_| A ZB|| E(z2) 
I(a) 1 |Z C D || Iz) (22) 
or 
W(21) = T(21 ,22)-W (22). (2b) 


can exist for each mode.” The input side is taken at 2; and the output 
side at 2, as in Fig. 1. The quantity Z is the characteristic impedance 
associated with the transmission medium. The directions of E and I 
are defined so that when the phase difference between EF and I falls in 
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NETWORK, 
T 





Fig. 1 — Generalized linear two-port network for single mode representation. 


the first and fourth quadrant, power is flowing in the direction 2; — 22, 
while for the second and third quadrant, the direction 2. > 2,. The 
choice of directions makes it possible to determine the result of cascading 
a number of sections by writing 


W (21) = T(2 22) -T (Ze 23) os T(2n-1 Bei) . W (Zn). (3) 


The complex quantities A, B, C and D are dimensionless and are 
independent of time in the steady state. In general, they are functions 
of frequency. For convenience, the characteristic impedance of the 
- transmission medium will be taken as unity, i.e., Zo = 1. The transmis- 
sion medium between planes z, and Zn4, will be referred to as a ‘“‘net- 
work.” The properties of the network are described uniquely by the 
quantities A, B, C and D. General relations among these quantities 
can be determined by specifying the transfer properties of the network. 
These are reviewed in detail in Appendix A and are described below. 7 
For example, a matched network which produces no reflection from 
side z, when terminated by the characteristic impedance on side 2n41 
can be written 


A B 


BA (4) 








in which A and B are, in general, independent complex parameters. 

A reciprocal network is characterized by the relation AD — BC = 1. 
A reactive network has the transformation 
a Jp 


IV (5) 








in which the four independent parameters a, 8, y and 6 are real. A recip- 
rocal reactive network, in addition, has a6 + By = 1, so that only three 
of the parameters are independent. A matched reciprocal reactive 
network has a = 6 and 8 = 7, so that only one parameter is independent. 
The network of this type can be written 


{+ The Appendix is included because there is no single convenient reference. 
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cose jsing (6) 
jsng cos¢g 





in which the real parameter ¢ is the phase shift from 2, to 2n4:,. A length 
of transmission medium is an example of a matched reciprocal reactive 
network. For this case, g = 2rv(Zn41 — 2n)/c’, in which c’ is the phase 
velocity of the radiation. 

The most general unmatched reciprocal reactive network, which 
has three independent parameters, can always be characterized as 























a je cos¢gi jsing,:||N 0 COS G2. 7 SIN gy 
. == ° —1 + (7) 
jy 6 jJsing: Cos ¢ O N 7 SIN G2 COS ge 
The network 
N 0 
0 Nw (8) 








is known as an ideal transformer of turns ratio N. Thus, the most 
general unmatched reactive reciprocal network, aside from phase shifts 
¢1 and ¢g , is an ideal transformer. 

A resistive network produces no phase shift and can be characterized 
as having A, B, C and D all real. A matched reciprocal resistive network 
has only one independent parameter and can be written 
cosh 6 sinh | 


sinh @ cosh @ (9) 





The matched reciprocal resistive network is known as an attenuator. 
It is shown in Appendix A that the power attenuation is given by 
exp —26. The parameter @ is referred to as the attenuator line length. 
The most general matched reciprocal network (resistance plus reactance) 
has two independent variables and can be written 


cos (g — j@) jsin (gy — 76) 
jsin (g — j@) cos (yg — j@) 





(10) 
cosh @ sinh 6 


sinh @ cosh @ 


cose jsing 
jsing cos¢g 














Therefore, the most general matched reciprocal network consists of an 
attenuator with phase shift. Note that the attenuation and phase shift 
commute. As a result, a matched network with distributed attenuation 
and phase shift can be lumped into a network with attenuation in 
cascade with a network having only phase shift. 
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The transmission factor or transmissivity of the network, denoted as 
L, is shown in Appendix A to have the value 


4 
~ TAF BTCHDE 


For a matched reactive reciprocal network as in (6), L = 1. The factor 
LI is also known as the gain of the network. The transmission factor 
equals the ratio of power transmitted to power incident when the net- 
work is preceded and followed by matched terminations. 

The reflection factor or reflectivity of a network is given by 


R=iL|A+B-C-—D|* (12) 


The reflection factor is the ratio of power reflected to power incident 
when the input and output terminals are matched. For a matched net- 
work, (A = D,B=C),R = 0. 

The noise generated in a network can be represented by suitably 
chosen current and voltage generators 7 and e at the input to the net- 
work as in Fig. 2.” The network itself can then be considered as noiseless. 
For example, for a series resistor 7, the noise generators are appropriately 
t= Oand|e| = [4(v)dvr]* in which 


L (11) 


hydy 
exp hv/kT — 1 


is the thermally generated noise power in a frequency range dv centered 
at frequency v when the resistor is at temperature T. The phase of e 
is a random variable. For a shunt resistor, 7, the noise generators are 
e = Oand |i| = [4p(v)dv/r]*. For reactive networks, e = 7 = 0. 

For an arbitrary network containing lossy elements, the appropriate 
values of e and 7 can be expressed in terms of the components of the 
transformation matrix, A, B, C and D. Since an arbitrary passive net- 
work can always be matched by suitable use of transformers and line 
lengths, placed on either side of the network, the arbitrary network can 


p(v)dvy = (13) 







NOISELESS 
NETWORK 


Fig. 2— Equivalent external current and voltage generators for noiseless 
network representation. 
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always be made to appear matched and resistive. It follows that an 
arbitrary lossy network can always be represented as a resistive matched 
network imbedded in reactive networks. The reactive networks on 
either side of the imbedded network are the inverse in reverse order of 
the networks required for matching the original arbitrary network. 

It follows, too, that if the appropriate values of e and 7 can be deter- 
mined for any resistive matched network, then the values can be deter- 
mined for the imbedded network. These values can then be transformed 
through the reactive networks at the input side of the imbedded net- 
work to represent the appropriate values at the input side of the original 
network. 

Thus, it is only necessary to have a general formula for e and 7 perti- 
nent to a resistive matched network. In Appendix B it is shown that the 
appropriate values of e and 7 for any matched resistive network are 
given by” 


I 


| g 


|i |? = AB4p(») dv 
et = et = 1(A” + B’ — 1)4p(v)db. 


le 
(14) 


The values of e and 7 without the factor 4p(v)dy will be referred to as 
the normalized values. The quantities, |e |’, |¢|” and e7* commute 
with a phase shift network (a length of transmission line). A value e 
following a transformer of turns ratio N becomes, at the input side of 
the transformer, Ne, while a current 7 becomes 7/N. Thus, the procedure 
in finding the values of e and 7 preceding any given network, T, is to 
find the appropriate reactive transformations of the form 


T = T(gi)-T(N) + Tye) -T(r)-T (es) - TCM) -T (es) (15) 


in which T(r) is the imbedded matched resistive network, and find the 
values of e and 7 appropriate to T(r) using (14). The values of e’ and 2’ 
appropriate to the input side of T are e’ = Ne and?’ = 1/N. 

In Appendix B, it is shown that the noise power into a matched load 
following any given network at uniform temperature 7’ (considering 
only the noise arising from the given network and ignoring the noise 
originating in the matched loads at the input and output side) is given by 


dP = L|e+i|*p(v)dp (16) 


in which Z is the transmission or gain factor for the network, given by 
(11), and e and z are the normalized noise generators at the input to 
the network. For example, for an attenuator with transmission factor 
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L, (16) yields 
dP = (1 — L)p(»)dv (17) 


as is shown in Appendix B. 

Since the noise parameters commute with a matched phase shift 
network, attenuation and phase shift can be lumped with respect to 
noise properties as well as transfer properties. 


IV. THE OPTICAL MASER 


The maser medium has the property that light passing through the 
medium once is amplified by a factor Gi(v). In addition, the light under- 
goes a phase shift, g(v). Thus, the matched maser medium can be 
characterized as an attenuator with a transmission factor G in cascade 
with a matched reciprocal phase shift network. 

Since the spontaneous emission from the maser medium can be con- 
sidered as thermal noise,” the spontaneous emission power radiated into 
a given mode by a uniform maser medium should be given by 


dP(v) = 1 — Gi(v)|p(»)dy (18) 


as follows from (17). Since p(v) = hv/(exp hv/kT — 1), the question 
naturally arises as to what temperature to associate with the maser 
medium. In particular, one wonders whether a noise formula like (18), 
which is valid for passive networks in thermal equilibrium, can be used 
when the maser medium is active, i.e., when G; > 1. Normally, the 
radiation temperature of the uniform maser medium is defined by the 
Boltzmann factor’ 


no(v)/ny(v) = exp —hv/kT (19) 


in which ne, and 7, are the densities of upper- and lower-state atoms, 
respectively. Using (19) as the definition of maser temperature, it 
follows that (18) is precisely correct for a uniform medium. 

To illustrate, one can write for the emission power, dP, into a given 
mode in a frequency range dy, along the z-axis 


ddP/dz = hywdP(n, — m) + 4dw.hyng (20) 


in which w; is the probability per unit intensity per unit time for stimu- 
lated emission into the mode and dw, is the probability for spontaneous 
emission in either direction for the same mode into frequency range d». 
The population densities for the upper and lower maser levels, nz and 


516 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


m, are assumed constant with z.t Solving (20) subject to the initial 
condition dP = Oat z = 0 yields 


3(dw,/wi)(1 — Gi) 


Pe) = Cufma) =I 


(21) 
in which 
Gi = exp hyw;(ne — m)2z. (22) 


Since the probability for stimulated emission is related to the probability 
for spontaneous emission into frequency range dv by 


dw, = 2w;hvdv (23) 


for a given mode, (21) and (18) are identical.{ It follows that (18) 
correctly accounts for the spontaneous noise into a single mode, so 
long as one writes p(v) = hv/(n/n2 — 1). It also indicates the applica- 
bility of the formalism described in the preceding section to maser media. 

The fact that the maser temperature 7’, as defined by (19), and p(y), 
as defined by (13), are negative should not distract the reader from the 
. more significant fact that (18) or (21) correctly predicts the noise 
power emitted by the maser medium. This quantity is never negative, 
and varies smoothly as 7’ goes from positive to negative values. 

The noise output from the maser can be calculated using (16), and 
this will be the aim of the following analysis. It is worth noting that the 
only significant parameters characterizing the medium, assuming that 
the maser medium is uniform and matched, are the total single-pass 
gain, G, , the population ratio and the phase shift through the medium. 
The effective attenuator line length, 6, is given by the expression Gi = 
exp —20. 


V. THE REGENERATIVE OPTICAL MASER OSCILLATOR 


The mirrors forming the optical cavity, being reactive (except for a 
small absorption loss which will be neglected) and reciprocal, can be 
characterized as ideal transformers. The phase shift associated with the 
mirrors on the cavity side can be added to the single-pass phase shift 
of the maser medium. The mirror phase shift external to the cavity is 
not significant to the problem at hand. 

} This implies no saturation or uniform saturation. 

} Equation (28) is equivalent to the statement that the total rate of spontane- 
ous emission is related to the total rate of induced emission per unit intensity by 


dw, = w,8rhrdv/c?, since the total number of modes per unit volume per unit 
frequency interval is given by 87v?/c3. (See Ref. 16.) 
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Since the reflection factor for an ideal transformer is, from (12) and (8) 
R = (N? — 1)°/(N* + 1)’ (24) 


it follows that the equivalent turns ratio for a mirror of reflectivity R 
is given by 


N? = (14+ R)/(1 — R). (25) 


The transmission factor L,, for the maser cavity with unequal mirrors 
of reflectivity R and R’ is obtained by combining the cascade of trans- 
former with turns ratio NV, attenuator with loss parameter @ = —31InGi, 
transmission line with phase shift ¢ = 2rvL/c’ + constant, in which 
2L/c’ is the equivalent round-trip time,{ and transformer with turns 
ratio M™*, M? = (1 + R”)/(1 — R"), as in Fig. 3. The noise power 





Fig. 3 — Equivalent circuit for maser cavity with mirrors of unequal reflec- 
tivity. 


from one end of the maser is given by (16) 
dP = Ly |Ne + N%|*p(v)dy (26) 
where the normalized noise generators for the maser medium are given by 
|e |” = |¢|* = cosh @ sinh 8 
ok * 1 2 - 12 
et = et = 4(cosh 6+ sinh’ @ — 1) 


as follows from (14) and (9) 
The cascade of mirror, maser medium and mirror takes the form 


N 0 cos (py — j@) jsin(y — jo)| |M™ a 


(27) 


0 N*| |jsin (ve — j@) cos (vy — 76) 0 
(N/M) cos (g — j0) NMjsin (g — 76) 
(NM)~j sin (g — j@) (M/N) cos (g — jé) 


{t The phase velocity c’ is a function of » and G; by virtue of the anomalous 
dispersion of the maser medium. 











(28) 
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Using (11), the transmission factor is given by 


4 
i> Ne Neo =) 
+ (MN + (MN) sin (p — 39)? 
(29) 


~ [= RRAGY/Ad — Rk) — RY] 
+ [4R:R2/(1 — R)(1 — R’)] sin? o 
Note that in the limit R = R’ = 0 (no mirrors), Lm = G1 as would be 


expected; while in the limit R = R’, G, = 1 (a transparent maser 
medium ) 


1 
~ 1+ (48/0 — Ry sin? g 


which is the transmission factor for the Fabry-Perot or optical cavity 
surrounding the maser medium.’"’ Equation (29), or versions of it 
with R = R’, has been derived before.” In these cases, however, it had 
been necessary to assume that the maser medium uniformly fills the 
region between the mirrors. No such restriction is necessary. 

The noise power | Ne + Nz |* has the value, using (27) 


|e + NG |? = (N? +N) |e |? + 2ei* 
= (N’ + N~”’) cosh @ sinh 6 (31) 
+ (cosh 6 + sinh’ @ — 1) 


Le (30) 


which after some manipulation yields 
|Ne+ Ni? = (1-G)A4+ RG)/GA1-R). (32) 


Combining (26), (29) and (82), the noise power in frequency range 
dv leaving the maser cavity through the mirror FR’ can be written 


(1+ RG)(1 — RB) — G)p)dv 
(1 — RIRIG,)? + 4RIR2G;, sin? o * 


In the vicinity of a cavity resonance at frequency », the phase shift 
¢ differs from some multiple of by an amount Ag = 2r(v — »)L/c’. 
The free spectral range of the cavity mode at frequency » is the range 
y —C/4LD Sv %y+ c'/4L as illustrated in Fig. 4; the mode spacing 
is c’/2L. Thus, the total noise power leaving the cavity through R’, 
associated with one cavity mode, is obtained by integrating dP over 
the free spectral range of the mode, yielding 


dP = (33) 
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vote’ /4L 
pe PO) Cr dy 

i Co + C3 sin? 2r(v — )L/c'” (34) 

Yo—c L 
With the substitution g = 24(v — »)L/c’, (34) can be written 
; +7/2 
= c Ci de 
P = pl) oF | CaF Grint 9 (35) 


in which the quantities C; , C, and C3 are given by 
Ci = [1 + RGi)(1 — RB) — Gi), 
C,= (1 —- PRAY 

and 
C; = 4R'R"G, . 


Since the integrand is large only in a very small range of frequencies 
near vo, it can safely be assumed that p(v), G; and R are constant with 
frequency and have their values at v = ». 

With this approximation the integral has the value 7C,/[C2’ + CC.) 
(see Ref. 18), yielding 


P(R’) = [(1 + RG&)(1 — R’)/(1 — RR’'GY)] 


(36) 
(1 — Gi) p(») (e'/2L). 





Vo-C'/4L Vo . VotC/4L 
—-_= 


Fig. 4 — Free spectral range of cavity mode at frequency vo. 
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' From (18) it may be noted that [1 — Gi(v0)]p(%)c’/2L is the spon- 
taneous emission power that would be emitted by the maser medium, 
in the absence of the cavity, into the spectral range c’/2L if the gain 
were constant over that range. When G; > 1, the spontaneous emission is 
enhanced by stimulated emission. The noise power 


[1 — Gi(v0)|p(%)e"/2L 


will be denoted as P,(%). The noise power leaving the cavity through 
mirror R’ is, therefore 


P(R’) = PA + RGi)(1 — R’)/(1 — RR'GY). (37) 


The power leaving the cavity through mirror F# is given by (37) with 
R and R’ interchanged. The total power leaving the cavity is 


P,= 2P,{1 — RR’G, + 4(G, — 1)(R+ R’)]/{1 — RR’GY| (38) 


so that the cavity enhances the spontaneous emission by the factor 
following 2P, . 

It should be noted that the integration of (36), which leads to (37) 
and (38), is valid even if G:(v) varies over the range c’/2L, so long as 
the frequency range over which G;, has a significant variation is large 
compared to the spectral range of the noise power. This will always be 
so, providing the natural linewidth of the transition is large compared 
to the spectral range of the noise, independent of whether the transition 
is homogeneously or inhomogeneously broadened. Equations (37) and 
(38) are valid even when the gain profile is saturated, providing the 
maser medium is uniformly saturated with respect to the axial direction. 
In general, the saturation will tend to be uniform, assuming uniform 
pumping, since the power in the cavity tends to be uniform with length. 
For very high-gain maser media the latter statement is not valid. 

It should be noted that (38) contains an implication which is not 
immediately obvious. In the limit ny = ne with G; = 1 and P, finite, 
(38) states that P, = 2P, independent of R or R’. Thus the total 
spontaneous emission noise power leaving the optical cavity is inde- 
pendent of its bandpass. The spectral distribution of the noise is altered, 
however, to correspond to the bandpass. t 

The preceding results can be used to determine the linewidth of the 
oscillator. The spectral range or bandwidth of the noise power is ob- 
tained from (33) by determining the frequencies at which dP falls to 


t In the limit c’/2L very much less than the inhomogeneously broadened line- 
width, (88) implies that the total spontaneous emission power into all modes is 
independent of the mirror reflectivity since G; = 1 over the entire line. 
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half its value at » = »). This occurs at frequencies », defined by 
4RIR?G, sin? 2a(vy — v)L/c’ = (1 — R'R"G)’ (39) 
yielding a spectral width Av = 2( — v9) given by 
Av = (c'/Lr) sin? 4(1 — R?R?G,)/(RR'GL)* 
ee (c'/2Lr)(1 — R?R?G)/(RR'GY)*. (40) 


Equation (40) can be rewritten by substituting for 1 — (RR’ \'@, the 
value given by (88) 


1 — (RR/G?)’ = 2(P./P.)[l — RR'G: + 3(G: — 1)(R + R’)I/ 

[1 + (RR’G;)'), 
The resulting expression is a function of the single-pass gain G;. For 
the strong oscillator the gain saturates at a value such that RR’G) 
differs from unity by a vanishingly small amount. Thus it is expedient, 


in the expression for Av found by substituting (41), to write for the 
saturated single-pass gain 


G, = [1 — (1 — (RR'G?)')1/(RR') (42) 


and then to replace 1 — (RR’G,’)* by its approximate value obtained 
from (41). In the reiteration, the approximate value can be written 
(R} +R")? (1 — (RR')') — QehvAv. 

4(RR’)} (RR’)* (1 — mi/ne)P: 
which follows by substituting in (41) the value G; = (RR’ )~? and re- 
placing P, by its value (1 — Gi)(c’/2L)hv/(m/n2 — 1). The cavity 
bandwidth Ap, is given by (40) with G, = 1. 

Combining (40-48) and performing the necessary algebra yields 


Av = 2n(hv/P,)(Ave)?(1 — m/nz) all + (ahvAr./P;) 
“(1 — m/m)™+ (— ale + 3) +14 2)/x*} 


(41) 


1 — (RR'GY)' (43) 


(44) 


in which 
z= (R*? + R")*/4(RR’)' (45) 


is a term which is identically unity when R = R’ and remains close to 
unity even for # and R’ differing by a factor of ten, and the quantity 
a = (RR’)'. 

When hvAv./P; <1 the linewidth is 


Av & an (Av,)” (1 —- n/N) (46) 
t 
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and differs from the well-known Schawlow-Townes’ formula by a factor 
of two. The correction term (1 — 2:/n2)~ approaches unity for the 
ideal maser (m/n2 = 0). This term has also been found by Shimoda.’ 
It should be noted that 7:/nz2 is the saturated value and in general may 
be very difficult to evaluate. An approximate value may be found by 
setting the single-pass gain given by (22) equal to the saturated gain 
given by (42). Extracting the appropriate value of ni/ne requires a 
knowledge of the transition probability, the time constants for the 
upper and lower states and the mirror reflectivities. The degeneracy 
of the upper and lower levels should also be taken into account. 

For the 6328 A gas maser, typical values for a one-meter-long dis- 
charge are P; & 10° watts/mode, Av, & 10° cps and m/n2 = 0.98, 
yielding a linewidth of 10~ eps. 

For the semiconductor optical maser, (46) predicts a linewidth of 
approximately 5 X 10° cps, taking L = 0.06 em and P,/P; & 10”. 
The latter numbers are taken from the data of Quist et al.’* The cor- 
responding wavelength range is 10” A. This estimate is probably a con- 
servative lower limit because of the neglect of internal losses in the 
derivation and the fact that the internal losses are significant in the 
actual device. The lowest observed linewidths are about 107 A. The 
relatively large linewidths arise from very short spontaneous emission 
lifetimes (<10~° sec). The relatively large amount of enhanced spon- 
taneous emission available produces saturation at small values of P;/P; . 

As has just been shown, it is possible to write formal expressions for 
the power output and spectral width of the noise emitted in a given 
mode. Since the power, and hence the spectral width, depend on the 
saturated single-pass gain, it is necessary to take the dynamic prop- 
erties of the maser medium into account. This is illustrated graphically 
in Fig. 5, in which the curve of P vs Gi as represented by (88) is shown. 
Also shown are a curve of Av and three dashed curves representing the 
dynamic properties of the maser medium. The latter curves may be 
found by solving the rate equations for the maser medium to obtain a 
relation between the power taken from the maser medium and the single- 
pass gain. When large power is taken from the maser medium, G; must 
approach unity, since the population inversion must approach zero. 
When the noise power taken from the maser medium approaches zero, 
the single-pass gain approaches its small signal value Gio. The curve 
representing the dynamics of the maser medium intersects the curve 
representing the static characteristics of the cavity at some value 
G, < (1/RR’)*. This is the operating point of the maser. The value of 
G, at the operating point determines the value of Av. 
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Avy. 





\ 
OPERATING 


POINTS 





(RR')V* 
G) —_> 


Fig. 5 — Noise power vs single-pass gain as represented by (88). 


For a gas maser with small single-pass gain, the dynamic properties 
of the maser medium can be shown to be controlled by an equation of 
the form” 


Gi mw + kol/(1 + KP of Ps) (47) 


in which ko is the small signal gain parameter for the mode in question, 
1 is the active length of the medium, «/P, is a saturation parameter 
dependent on the Einstein A and B coefficients for the maser levels and 
P, is the power in the cavity. In most maser media « < 1, so that Gi 
varies quite slowly with P./P, . It ee be noted that P, is essentially 
constant and can be evaluated for G; = 

The gain parameter ko varies with ote current. or low cur- 
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rents, ko is proportional to current.” Thus, as the discharge current is 
increased from zero, the output power increases. From Fig. 5 it would be 
expected that as the discharge current approaches a value for which 
1 + kol approaches (RR’)~?, the output power would show an extremely 
sharp rise with current. This can be illustrated by writing for G; as a 
function of discharge current J, taking R = R’ for convenience, 


Gy 1+ [ — R)/R\/To)/C1 + «P./Ps) (48) 


in which Jo is the current for which G,R = 1 in the absence of satura- 
tion. From (388), taking P, = P,/2(1 — R) 


P./P, = 1/(1 — RG). (49) 
Solving (48) and (49) for P./P, yields 
P./P, = [AI/Io + « + W(AT/Ig + x)? + 4x]/2k (50) 


in which AI = I — I). Note that when (AI/Ip) + x = 0, P./P; = x’. 
When (AI/Io) + « has the value «*(f — f), P./P; has the value fx’. 
Therefore, the change in P,/P, by a factor f is larger than the change in 
AI/Ip of x’(f — f°) by a factor x *, Thus, when « < 1 the power output 
shows a very sharp threshold with current. The value of « is of order 
10° for the 6328 A gas maser. 


VI. THE NONREGENERATIVE OPTICAL MASER OSCILLATOR 


Some maser media have such large single-pass gain that spontaneous 
emission originating at one end can be amplified sufficiently to saturate 
the maser medium at the opposite end. For example, the 3.39 y» transi- 
tion in neon, in a helium-neon gas maser, has gains of order 50 db/meter 
in small-bore tubing. The saturated output power is in the 1-10 mw 
range. Xenon at 3.5 uw has even larger gain.” Under these circumstances 
the maser can behave as a saturated oscillator without an optical cavity. 
The extreme line narrowing characteristic of the cavity oscillator will 
be lacking, but the power levels and directionality will be comparable. 

This type of oscillator is characterized by a peak output always at 
the line center, independent of temperature and any physical dimension, 
line narrowing over the inhomogeneously broadened line and an ex- 
tremely stable output. The power per steradian per cycle will be much 
greater than conventional light sources, and so these structurally simple 
oscillators may be very useful as frequency standards and for calibra- 
tion purposes. 
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Some idea of the line narrowing possible with this type of oscillator 
can be obtained by reference to (18), which describes the noise power 
emanating from one end of the unsaturated maser medium in one mode 
with no mirrors at either end, while (33) with R’ = 0 gives the noise 
power when one end has a mirror, but the other does not 


dP = [1 + RGi(»)][1L — Gil) p(y) dv. (51) 
When & = 0, (51) properly yields (18); however, when R = 1 
dP = (1 — G’(v)|p(v) dv. (52) 


Since G,’ is the gain of a maser of twice the length of a maser of gain 
G, , the perfectly reflecting mirror placed at one end serves to double the 
effective length of the maser medium. 

Assuming no saturation, so that the gain can be written 


Gi(v) = exp k(v)l (53) 


in which / is the effective length of the medium, and assuming further 
that the gain parameter k(v) has a Doppler profile 


k(v) = ko exp — [2(v — v)/Avpl” In 2 (54) 


in which » is the frequency of the line center and Ay» is the full Doppler 
width, the spectral width at half power of the spontaneous emission 
Av can be determined by writing 


1 — exp [kol exp — [Av/Avp]’ In 2] = 34[1 — exp hol]. (55) 
In the limit kol > 1, (55) can be solved for Av/Avp to yield 
Av/Avp = (Kol). (56) 


The 3.39 uw maser is saturated by its own spontaneous emission by gains 
of order 80 db, so that the maximum possible value of kol before satura- 
tion is about 20, corresponding to 1 } 2 meters. This yields Av & Avo/4.5. 
The Doppler width at 3.39 » is about 300 me, yielding Avy } 70 me. 
It would be possible to decrease this value somewhat by using several 
lengths of maser media separated by attenuators to prevent saturation. 
The slow dependence on / exhibited by (56) indicates the impracticality 
of this scheme in achieving any more than a factor of four decrease in 
linewidth unless kp can be increased drastically. Fortunately, there is 
evidence that this can be done, and the nonregenerative oscillator may 
turn out to be an extremely interesting device. In addition, it can be 
tuned by application of magnetic fields. 
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VII. CONCLUSION 


The output properties of an optical maser oscillator have been de- 
rived subject to the supposition that the oscillator is a saturated ampli- 
fier of spontaneous emission noise. The most significant new results 
concern an expression for the linewidth of the oscillating maser which 
differs from the commonly accepted value. Some techniques, well-known 
in the microwave art and used only in a limited way in optics (stratified 
media), have been generalized to apply to the transmission and noise 
properties of optical masers. 
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APPENDIX A 


Network Representations 


The basic network of interest will be a linear two-port network which 
will be represented schematically by Fig. 1. The major concern here will 
be the relationships between the quantities J, and Z, at port 1 and I 
and EK. at port 2. The linear relationship among these quantities will be 
represented by the transformation matrix 




















Ey; _|A B| |E, 
Fe Co Dede (57a) 
which will be abbreviated by 
Ww, = TW. (57b) 
in which 
W; = a an eee (58) 
q; 
and 
A B 
T= | a f (59) 


Note that the direction of positive current flow is defined to be into the 
network at port 1 and out of the network at port 2. This choice sim- 
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plifies the discussion of cascaded networks. It is clear that two net- 
works in cascade can be represented by W, = Ti:-W,. = T,-Te- WW. 
The product T,-Tz follows the usual rules of matrix multiplication. In 
general, T;-T2 ~ T.-Ti; that is, the two networks do not commute. 
For a cascade of n networks 


Wy, = T.:Te eee Ta: Wry. (60) 


The determinant of the transformation will be a quantity of interest 
and will be defined as A = AD — CB. The inverse of (57) is defined by 


Ww, = TOW, (61) 
in which 
1 D —-B 
TT = a (62) 
—C A 


is the inverse transformation matrix. Reference will sometimes be made 
to the exchange network T, which is merely the network T with its 
terminals exchanged so that port 2 becomes port 1 and vice versa. The 
components of T are obtained by writing W. = T-w, and noting 
that for the exchange network the quantities FE, , I, become Ey’, —Iy’ 
and E,,1, become H,’, —I,’. As a result, the correct description is 
given by 


Ey D B|\E 
1 
I C Al [LD 
and 
1 D B 
C A 
The net power into port 1 is given by 
Pi = 3(E* + EY) = Dw, (65) 
in which W is the complex transpose of UW and 
01 
z= 1 0 : (66) 





Likewise, the net power out of port 2 is given by 
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P, = AW, DW, . (67) 


Other transformations will be introduced as they are needed. In the 
following, certain types of networks will be characterized in terms of 
relationships among the components of the transformation matrix, 
ie., A, B, C and D. These networks will form the basic ‘‘tools” of the 
analyses. 


A.1 Matched Networks 


If one of the ports of the network is terminated in a matched load 
and it is then observed that the input impedance of the other port is 
matched to the line, the network is said to be matched. First, consider 
the input impedance of the network in Fig. 6. The line is assumed to 
have unit characteristic impedance. The quantities B and C are thus 
normalized with respect to the characteristic impedance and admit- 


E, 


Zo= 1, 


Fig. 6 — Terminating impedance for matched network. 


tance of the line, respectively. Writing EL; = AL, + Bl, , J, = Cl, + DI, 
and 


Zinputay = Lh/I, = (AZ, + B)/(CZ, + D) (68) 


in which Z. = F/I, is the terminating impedance. The network is then 
matched if Zinputa) = Ze = 1, yielding the requirement A + B= C+ D. 
Likewise, the exchange network must also be matched, requiring 
D+ B=C4 A. The two requirements yield the relations A = D, 
B = C, and for a matched network 

A B | 


Epo A 


(69) 





A.2 Rectprocal Networks 


A reciprocal network has the same transmission properties in either 
direction. It follows that for a reciprocal network T and T must have 
the same transmission factor and phase shift, which from (84) and 
(72) implies A = 1. The simplest nonreciprocal network can be written 
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2 








A’ 0 
with A ~ 1; and all nonreciprocal networks can be written 
_|4 Bi_|a° 0] |4/a? B/at| _ 
ca ae 7) laa Ee C1 DIR oD) 




















The reciprocal network T, is known as the reduced network; all the 
nonreciprocity resides in T, . The network T, , known as the abstracted 
network, commutes with all networks. 


In general, A is complex and can be written A = | A | exp 7 2¢. Since 
A |? exp jg 0 
4 | } : 72 
0 | A |* exp jp a 








and the exchange network 


|Al*exp jp sO 
0 | A|™ exp —je 





Te = 





(73) 
it follows that argument ¢ is the nonreciprocal phase shift. 


A.3 Reactive Networks 


Since a reactive network is lossless, it would be expected that P. = P,. 
It follows that W.5W, = W.TITW,, so that for a reactive network 
TT = 2. Performing the indicated matrix multiplication yields 

A*C + AC* A*D+C*B i 1 
AD* + CB* BD* + B*D 1 0 
It follows that A*C and BD* are imaginary and A*D + C*B = 1. 


Note that multiplication of A*D + C*B = 1 by CB* yields CB* 
(A*C)(B*D) + |C ?| B |’, which is real. Likewise, 


AD* = | A [’| D+ (AC*)(BD*) 


I 


‘ (74) 








is real. If the reactive network is reciprocal so that A = AD — BC = 
1 = (AC*)(D/C*) — D*B(C/D*) = (A/B*)DB* — (B/A*)A*C, it 
follows that both D/C and A/B” are imaginary. Since AC” is imaginary 
and C*B is real, A/B is imaginary. Thus B/B* is real, which can only 
occur when B is real or imaginary. It follows that B and C are real 
(or imaginary) while A and D are imaginary (or real). The question 
of which set to choose real is decided by noting that the idemfactor 
(no network) is a reactive reciprocal network. Thus, the appropriate 
choice is A, D real and B,C imaginary, and the reactive reciprocal 
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network can be written 
a jp 
jy 6 








(75) 
for which a6 + By = 1. If the reactive reciprocal network is also matched 
so that a = 6 and B = y, the network can be written 


cosy jsing 
jsng cose 








(76) 


in which the parameter ¢ is known as the angular length or phase shift. 
Note that the input impedance for this network is 


Zo +7 tang 


1+ Zejtang (77) 


Zinput(1) = 


which is recognizable as the impedance transformation for a transmis- 
sion line of angular length ¢ and unit characteristic impedance. 

Next, certain basic passive circuit elements of interest will be charac- 
terized. . 


A.A Sertes Impedance (Fig. 7) 
Since J; = Jz and Ly = InZ + EH, , the transfer matrix is given by 





1 Z@ 
eo R 1 (78) 
A.5 Shunt Admittance (ig. 8) 
Tor the shunt admittance, ; = Hy, and I, = EH.Y + I. ; thus 
1 0 
r= | Yi | : (79) 


Both networks are reciprocal but not matched. 





Fig. 7 — Series impedance. 
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Fig. 8 — Shunt admittance. 


A.6 Ideal Transformer of Turns Ratio N :1 (Fig. 9) 
For the transformer Z, = NE, and I, = I2/N,. Thus 








N O 
P= 0 No (80) 
oe Ge 
| : | 
E, f {e. 
| | 
Le ast pa 4 


Fig. 9 — Ideal transformer. 


A.7 Network Transmissivity 


Consider a general two-port network which has a matched generator 
on side 1 and a matched load on side 2. The cascade of networks may be 
represented schematically as shown in Fig. 10, yielding 




















E,|_|1 1| |A Bl |1 oO] |e 
P| =| 1! pelle 0 |" oe 
Ig—> { Is=0 





Fig. 10 — Circuit for two-port network with matched generator on side 1 
and matched load on side 2. 


Performing the indicated matrix multiplication yields 


| Zo pe eo a al 


‘I, C+D D 0 2) 
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The network transmission factor is defined as the ratio 
L = | E.|?/| Bol? (83) 


in which F is the voltage that would be measured if the network were 
absent. In the latter case, the appropriate two-port transformation 
must be the identity matrix, so that A = D = 1, B = C = 0. Hence 
2 7 2 
rs a . (84) 
|A+B+C+D/> 4, |A+B+C+ DP 


A transmission factor greater than one implies gain. The network trans- 
mission factor for a matched network is 


L=4|244+2B\|7*=|A+B|~ (85) 


A.8 Network Reflectivity 


When a network is followed by a matched load, the input impedance 
to the network is given by Zinputs = (A + B)/(C + D). The power 
reflectivity, R, is given by 
p — | (Fim — DP _ |A+B-C — DP 

(Zinput an 1) | A ++ B ac C oF D |? (86) 
=iL|A+B—C—DP 


in which LD is the transmission factor of the network. Note that for a 
matched network, A = D, B = Cand R = 0. 





A.9 Matched Attenuator 


An attenuator is a nonreactive reciprocal device. Thus, a matched 
attenuator must satisfy the conditions A = D, B = Cand AD — BC = 
1, with A, B, C and D real. This is automatically satisfied by writing 


cosh @ sinh @ 
sinh @ cosh @ 


: (87) 








The parameter @ is referred to as the line length of the attenuator. The 
attenuator commutes with a matched reciprocal reactive network (a 
transmission line, for example) and the resultant network 


cosh 6 sinh @ | 
sinh 6 cosh@ 


cosg jsing 
jsng cose 








(88) 
cos (g — j@) gsin (¢ — 30) 
jsin (g — j@) cos (» — 76) 
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is the representation for an attenuator with phase shift or angular length 
g. From the fact that a lossy transmission line would have a phase shift 
of the form exp —j(¢ — j@) = exp —0@ exp —jy, it would be expected 
that the transmission factor is given by exp —26. For an attenuator 
the transmission factor is obtained by using (85) and (87) 


L = | cosh @ + sinh 6|~” = exp —20. (89) 


Thus, the attenuator matrix is specified completely by knowledge of 
its transmission factor. 


A.10 Isolator 


An isolator is by definition nonreciprocal although it is matched. 
Thus, an isolator with forward transmission unity and reverse trans- 
mission exp —26 can be synthesized by cascading an attenuator of line 
length 6/2, having a transmission exp — 6, with a nonreciprocal network 
with transmission exp +6 in the forward direction and transmission 
exp —9@ in the backward direction. In its most simple form, the non- 
reciprocal network (70) is given by 











exp — 6/2 0 
0 exp a ; (90) 
Thus, the isolator can be represented by 
T= jexp —6/2 0 cosh 6/2 sinh 6/2 | 
ae 0 exp —6/2|| sinh 6/2 cosh 6/2 1) 


exp —0/2 cosh 6/2 exp —6/2 sinh 6/2 
exp — 06/2 sinh 6/2 exp —6/2 cosh 6/2 








and the transmission factor is unity in the forward direction and exp —20 
in the backward direction. An ideal isolator is one for which the line 
length 6 approaches infinity, yielding 


T= 





dol vole 


: ow) 


NIH ple 


APPENDIX B 


Nowsy Networks 


In the following, the source of noise will be considered to be spon- 
taneous fluctuations which arise because of the thermal properties of 
the material. At extremely high frequencies, the thermal noise is more 
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commonly called black-body radiation. The noise spectrum will be 
characterized by the following statement of Nyquist’s theorem: The 
noise power available per mode at frequency »v in a small frequency 
interval, dv, is given by 


dP = p(v)dv 
p(v) = hv(exp hv/kT — 1)7 


| 


(93) 


| 


for a passive circuit at temperature 7’. The constant k is Boltzmann’s 
constant (1.38 X 10°” joules/degree) and h is Planck’s constant (6.6 X 
10 * joules-sec). 

It is sometimes convenient to write the noise power in terms of equiva- 
lent rms voltage and current generators e and 7 as shown in I'ig. 11. 
The internal impedance of the voltage generator is zero and for the 
current generator it is infinite, and one can write 


le| = (4rp(»)dv)?, | é| = (4gp(»)dv)?. (94) 


In the following, both e and 7 will have the units of (power)? since r 
and g are normalized with respect to the line impedance and admittance. 
A systematic method of handling the noise produced by a network 
will be developed next. First, the following theorem will be stated with- 
out proof: Any passive noisy two-port network can be replaced by an 
equivalent noise-free network which has an added shunt current generator 
and series voltage generator at the input or output terminals which represent 
the noise contribution.” Therefore, any passive noisy network can be 
replaced by the representation shown in Fig. 2, in which the network 
is now noise-free but the noise appears from equivalent voltage and 
current generators at one of the terminals. A proof for the theorem can 
be given, and although it is relatively simple, the proof is lengthy. 
Next, a scheme for representing in a simple way the additional current 
and voltage appearing at the terminal will be described. The following 
technique is due to H. Seidel.” It is clear that one may always write 


KH, = AH,+ Bl, +e 
I; CH, + DI, + 7. 


io 


Fig. 11 — Noise-free representation of network with equivalent external 
voltage and current generators at one terminal. 


(95) 


I 
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The inclusion of the noise generators can be accomplished in an artificial 
but, as will be seen, highly useful way by writing 


iy A B é yp 
hj=|C D aij-} de}. (96) 
1 0 01 1 


This representation results from adding the trivial equation 1 = 0 + 
O + 1 to the set in (95). No new information has been added, but the 
bookkeeping advantages afforded by this change will become apparent 
shortly. 

Equation (96) will be the general representation for a passive noisy 
two-port network. The problem now is to learn how to characterize the 
nolse quantities e and 7 in terms of the properties of the network repre- 
sented by A, B, C and D. It will be seen that this can be done by com- 
paring the network to some simple network whose properties are known. 

First, it will be demonstrated that for any passive network, one may 
take A, B, C and D either all real or all imaginary. This is equivalent 
to saying that the network may always be taken to appear purely 
resistive. This is clear since the input impedance is Zj;, = (AZ, + B) 
(CZ, + D). If Z, is real, then it would be expected that a resistive 
network will have Zj, real also. The proof follows from the fact that the 
input impedance can always be made real with a suitable length of 
transmission line. In addition, with an ideal transformer one may 
match the input impedance to the line. It follows, therefore, that for 
noise calculations, one only need consider matched networks (A = D, 
B = C) with the ratio A/B real. 

First, consider a matched network which is cascaded with a transmis- 
sion line of arbitrary length 6. The latter network has no loss and has 
no source of noise. In addition, the transmission line can change only 
the phase but not the magnitude of the noise current and voltage 
generators. Consequently, one expects that for a matched network 


cos@ jsiné O|/A B e X X ecosé+ 1% cos 8é 
jgsin@é cos@ O||}B A t1=|X X ejsind+ 2cos 6 
0 0 1;/}0 0O 1 0 O 1 


has the same noise properties independent of @. Thus, it is required that 
|e|*> = |ecos6+isin6|* = |e|* cos 6+ |7|* sin’ 0 

+ e(ji)* + e* (ji) sin @ cos 8 
or 


(|e|> — |¢|*) sin’ 6 + j(ei* — e*%) sin 0 cos 6 = 0. 
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Choosing 6 = 7/2 yields 
Je|* = lal? (97) 
and it follows also that 
et = eu. (98) 


Next, the passive matched network at temperature JT will be cascaded 
with a shunt conductance at the same temperature and the open-circuit 
noise voltage at the input terminals will be determined. The network 
representation for the shunt conductance in the new formalism follows 
from Nyquist’s theorem as is shown in Fig. 11 


|i’ | = (4Gp(v)dv)? 















































A Be 1 0 0 
C.D. wiser 1-4 (99) 
0 01 001 
Thus, the network to be studied is as shown in Fig. 12 with 
iy A Boe 1 0 0 i 
0O;=|B Aidt G1 7|-10 
1 0 0 1 0 0 1 1 
A+ BG B Bi'+e|| Ff 
=|B+AG A Ai’ + 0 (100) 
0 1 1 
yielding 
E, = (A + BG)E, + Bi’ +e 
(101) 
0= (B+ AG)E, + A’ +7 
so that 
ae (Ad’ + 4) 
B+ AG 
= ses eee of . 4 
E, = BLAG (Ai +7) + Bi +e. 


At the open-circuited terminal at which the noise voltage E, is meas- 
ured, one must have from Nyquist’s theorem, (94) 


| Ey |? = 4Zinpurp(v) dv (103) 
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Fig. 12 — Passive matched network with shunt conductance for determina- 
tion of open-circuit noise voltage. 


since Zinput 1s resistive. Its value is 


Fine = 2 BG 
meu BAG’ 


Hence, it is required that 


(104) 





A + BG A+ BG\,,7,; ’ ° 

pan Ss el Se « CE 
(4 = Be) 4n(v)dp (4 + ay (Ar +7) + Bt +e (105) 
Remember also that |e |” = | 7 |” and et” is real. Since the noise arising 


from the shunt conductance G is independent of the noise produced by 
the network, cross terms like e7’, 7"i’ are zero, since their product repre- 
sents a time average which must be zero. Solution of (105) yields 
uniquely the values” 
|i]° =|e|* = AB 

ei = 1(A* + B’ —1) 
which are the desired relations measured in units of 4p(v)dv. The 
veracity of (106) can be established by direct substitution into (105), 


which yields an identity independent of the value of G. 
The equivalent values of e and 2 for an attenuator are given by 


[e|* = |2|* = AB = cosh @ sinh 6 = 3 sinh 20 
ei = 3(A?+ B’ — 1) = 4(cosh’ 6+ sinh’ 6 — 1) = sinh’ 8. 


(106) 


(107) 


The phase angle between e and 7° is cos tanh @. Since the phase angle 
is always positive, the implication is that the noise sources radiate more 
power toward the attenuator than away from it. This is reasonable, 
since the noise power leaving each end of the attenuator should be equal 
and the noise radiated toward the attenuator by the noise sources is 
attenuated before emerging from the output end. In the limit of large 
6, the phase angle approaches zero. 
The equivalent values for an isolator are given by 
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Je |? 


l 


|7 |)? = exp —0 cosh 5 sinh 5 = + exp —é@ sinh 6 


5] exp —6 (sinn? § ++ cosh’ $) — | = —le|* 
Therefore, the angle between e and 7 is always z, the implication being 
that the isolator radiates noise power away from the attenuator, i.e., 
only in the direction in which it attenuates. 

The noise radiated into a matched resistor by any network is ob- 
tained by considering the network shown in Fig. 13 


(108) 


er* 


I 





Fig. 18 — Noise radiated into matched resistor. 


























Ey 1 0 OJ}; A B efj1 O O}|F 

O/=/1 1 O;/C D ail 1 O}F0 

1 0 0 1//0 O 1)/0 0 14/1 
A’ B’ eli, 
=|C’ D’ wv}}0 ]. (109) 
0 O 41/))1 





The noise contributed by the matched input and output resistors is 

neglected since this is additive. The primed quantities result from matrix 

multiplication. The noise output dP is evaluated by noting that 
O0=C'E, +7 and dP=|F|?=|7|7/|C'|* 

Performing the matrix multiplication yields C’ = A+ B+C+D 

and 2’ = e + 2. Thus, the output noise power is given by 

le + 2) 4p(v) dv 
|A+B+C+D/ 


in which L is the network insertion loss [see (20)]. For a matched resis- 
tive network (attenuator) 


P= =Dle+ i? p(v) dy (110) 


le tal? = fel?+ |i]? + 2e* = (2QAB+ A’? + B’— 1) 
=[((A+ BY-1)=L"-1 
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which follows from (9) and (21). Thus, 


dP = (1 — L)p(»)dv (111) 


is the noise power in frequency range dv emanating from a matched 
resistive network. This result is well known in network theory. In 
optics, it is known as one form of Kirchhoff’s law. 
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The 80 Diperiodic Groups in 


Three Dimensions 


By ELIZABETH A. WOOD 
(Manuscript received December 10, 1962) 


The low-energy electron diffraction work of L. H. Germer, J. J. Lander, 
A. U. MacRae, J. Morrison and others is resulting in new information 
about surface structures. These three-dimensional structures have periodicity 
only in two dimensions. The 230 triperiodic space groups are not applicable 
to the solution of these structures. The 17 strictly two-dimensional groups 
do not admit the existence of a third dimension and may therefore not be 
appropriate for these structures which are not strictly planar. The useful 
space groups for these structures are the 80 diperiodic groups in three 
dimensions. 

Nowhere in the literature have these been put into a form convenient for 
use, as have the other two sets of space groups. This has now been done and 
the tables are available on request from the Circulation Manager, Bell 
System Technical Journal, Bell Telephone Laboratories, Incorporated, 
463 West Street, New York 14, N. Y. Sample tables are given in this paper. 


I. BACKGROUND 


Crystals grown under favorable conditions acquire an external shape 
whose symmetry has long attracted attention. Nineteenth century 
mineralogists systematically described the symmetry of these shapes in 
terms of symmetry operations. For example, the operation of rotation 
of a cube through 90° around an axis normal to a cube face brings the 
cube into a position indistinguishable from its original position. An 
operation that achieves this indistinguishability is called a symmetry 
operation. In this example the cube will present an identical appearance 
four times during a rotation of 360° around the axis, which is therefore 
called ‘‘an axis of 4-fold symmetry” or simply ‘‘a 4-fold axis.’”’ A cube 
has three 4-fold axes, four 3-fold axes (corner-to-corner) and six 2-fold 
axes (mid-edge-to-mid-edge) (Figs. 1a and b). Such axes are called sym- 
metry elements. The terms “tetrad,” ‘‘triad’”’ and ‘‘diad” are also used 
for them. 
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Another type of symmetry element is a mirror plane, across which the 
operation of reflection produces an object indistinguishable from the 
original. Such a plane through the center of a cube parallel to two 
opposite faces reflects the left half into the right half and vice versa; 
that is, the two halves are mirror images of each other. Since there are 
also diagonal mirror planes in a cube there is a total of nine planes 
(Figs. le and d). 

There is also a center of symmetry in the center of a cube which re- 
lates any feature located a given distance from it in one direction to an 
indistinguishable feature located the same distance away in the opposite 
direction. The operation is called inversion. 
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Fig. 1 — The symmetry elements of a cube: (a) the three 4-fold axes and the 
four 3-fold axes, (b) the six 2-fold axes, (c) three mirror planes parallel to the 
faces, and (d) six diagonal mirror planes. 
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An inversion axis combines the operation of rotation with that of 
inversion. The familiar regular tetrahedron which has neither a 4-fold 
axis nor a center of symmetry has a 4-fold inversion axis because after 
a rotation of 90° plus an inversion through the center point it occupies 
a position in space indistinguishable from its original position. A center 
of symmetry is equivalent to a one-fold inversion axis. 

Note that during the entire group of ‘‘operations” on the cube, one 
point (the center of the cube) remains unmoved. Another way of saying 
this is to say that all of the symmetry elements pass through a single 
point. This group of operations or the symmetry elements which repre- 
sent them therefore constitute the point group symmetry of the cube. 
When similar groups of operations are determined for all possible 
crystals, it is found that there are only 32 possible crystallographic 
point groups. 

The symmetry of shape is the outward expression of the inner orderly 
atomic arrangement of the crystal. Any property of any piece of the 
crystal must obey the point group symmetry even though the piece be a 
ground sphere a few tenths of a millimeter in diameter. 

When we consider in detail the crystal structure — that is, the posi- 
tions of the atoms relative to each other — we find that the symmetry 
elements occur at well-defined positions in space and do not all go through 
the same point. This is readily illustrated by Fig. 2, the projection of 
the structure of calcite (CaCO3) onto a plane normal to its 3-fold sym- 
metry axis. Note that the 3-fold axis cannot be randomly placed, normal 
to the paper, but must pass through the black spots representing the 
carbon atoms, and further that there is a 3-fold axis through every 
carbon atom. There are also mirror planes in calcite. We could make a 
3-dimensional model of the array of symmetry elements of calcite, 
and the operation of any symmetry element would shift every other 
symmetry element to a position indistinguishable from its original 
position. 

Such a self-consistent array of symmetry elements in space is called 
a space group. Since location in space (not orientation alone) is of signifi- 
cance here, two other kinds of symmetry operations become meaning- 
ful: operations which combine translation with either rotation or reflec- 
tion. The resulting symmetry elements are called, respectively, screw 
axes and glide planes. 

As in the case of point groups, the space groups are limited in num- 
ber. There are 230 possible space groups, i.e., 230 possible self-consistent 
arrangements in space of all the symmetry elements mentioned above. 

Diagrams of these are given in the International Tables for X-ray 
Crystallography (edited by Henry and Lonsdale, 1952; see References). 
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Fig. 2— The structure of calcite projected onto a plane normal to its 3-fold 
symmetry axis. 


The one appropriate to the structure of calcite is shown on the next 
page.* It is identified by the symbol R3c which states that the unit cell 
(the repeat unit of the structure) is rhombohedral in shape (22), that it 
has a 3-fold inversion axis (3) with a glide plane parallel to it in which 
the translation is in the c direction.t Of course the 3-fold axis operating 
on this glide plane generates two more. Additional symmetry elements 
which are found to exist whenever the stated symmetry operations are 
performed are also shown in the calcite space group diagram. 

The space group of a crystal can in many cases be uniquely deter- 
mined directly from x-ray diffraction data. Since, in any given space 
group, the possible atom positions will be related in a well defined manner 
by the symmetry operations, a knowledge of the space group is a very 
powerful aid in determining the arrangement of atoms in the crystal, 
i.e., the crystal structure. 

One could repetitiously extend the space-group symbols in the dia- 
gram, as we have the calcite structure in Fig. 2, by translation which 
would be in three dimensions if we were not limited to the printed page. 
(The translation vectors define the edges of the unit cell, the repeat 
unit of the three-dimensional] structure.) The three-dimensional lattice 
of translation vectors which would represent this operation is called 
a space lattice. There are only 14 such lattices possible. 

If we limit our attention strictly to two dimensions we find that, 
instead of 230 space groups, we have 17 plane groups and instead of 14 
space lattices we have 5 nets. Here the periodicity no longer extends in 
three dimensions (triperiodicity) but only in two dimensions (diperio- 
dicity). 

* Fractions on the diagram refer to positions of symmetry elements along the c 
direction (normal to the paper). The unit is the unit length of c, i.e., the c di- 
mension of the unit cell. 


+ For a very brief discussion of space group symbols see the Crystallographic 
Data section of the American Institute of Physics Handbook. 





No. 167. R 3 ¢ 







































































Trigonal 3m R32/c 
D3a 
a 
4 
as al al / x tA 2 1 4 1 
4 4 2 2 44 2 #2 4 4 
Symbols of Symmetry Planes 
Graphical symbol 
Symbol ee Nature of glide translation 
Normal to plane 
of projection 
c Axial glide J tr eeeeeeeee c/2 along z-axis; or (a+b+c)/2 along [111] on 
plane rhombohedral axes. 
n Diagonal glide | ----------- (a+b)/2 or (b+c)/2 or (c+a)/2; or (atb+c)/2 
plane (net) (tetragonal and cubic). 
Symbols of Symmetry Axes 
ete a iiate a 
Symmet Graphical Dea Symmet Graphical pi poabarnen 
Symbol Maris | aymbol | FSEE amg [mEON) axis” | symbol Sto ah 
the axis the axis 
1 | Rotation None None 2, | Screw. 6 c/2 
monad diad 
(normal to 
2 paper) 
1 | Inversion None 
monad 9 —— 
; (parallel to |a/2 or b/2 
2 | Rotation @ None paper) or (a+b)/2 
diad (normal to 
F :.paper) Normal to 
paper 
— 3 Rotation A None 
(parallel to triad 
paper) 
3: | Screw A. c/3 
triads 
32 A 2¢/3 
3 Inversion A None 
triad 











545 


546 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1964 


It is with still a third set of groups, the 80 diperiodic groups in three 
dimensions, that the present paper is concerned. 


II, DISCUSSION OF THE 80 DIPERIODIC GROUPS 


The International Tables for X-ray Crystallography (1952) 
(“ITXRC”’) give two different sets of space groups: the familiar 230 
triperiodic space groups and the 17 two-dimensional space groups in 
which all operations are confined strictly to two dimensions. In the 
latter set, any operation which admits the existence of the third dimen- 
sion, such as a two-fold axis lying in the plane, is forbidden. 

The existence of a set of groups which admit such operations, but 
still refer to arrays that are infinitely periodic in only two dimensions, 
was recognized by several authors at about the same time (Speiser, 
1927; C. Hermann, 1928; Alexander and K. Herrmann, 1928; L. Weber, 
1929; Alexander and K. Herrmann, 1929). These and subsequent authors 
(see references at end of this paper) have used a wide variety of nomen- 
clature, some giving some diagrams. C. Hermann gives point positions, 
but in many cases chooses a different origin and in some cases a larger 
cell than that given in ITXRC. This work and others contain errors 
and omissions and none of the authors has given the groups in the form 
currently used in the International Tables so that they could be con- 
veniently used for structure determination. This has now been done. 

Consideration of the restrictions imposed by the loss of periodicity 
in the third dimension leads to the exclusion of the following symmetry 
elements: (7) screw axes normal to the plane of diperiodicity, (77) glide 
planes with glide directions out of this plane, and (777) n-fold axes not 
normal to this plane, with n > 2. Since the upper side of our diperiodic 
array may be like or unlike the lower side, mirror planes, glide planes, 
two-fold rotation and screw axes may lie in the plane. 

It is possible to choose the 80 diperiodic groups in three dimensions 
from the pages of the existing International Tables for X-ray Crystal- 
lography by using some of the ‘‘Ist setting” monoclinic groups and some 
of the “2nd setting”? monoclinic groups as well as various orientations 
of the orthorhombic groups, without deletion or addition of any sym- 
metry operations. In the diperiodic-group case we always have a unique 
direction in the plane-normal. Placing this direction along each of two 
nonequivalent directions in a single (orthorhombic) triperiodic space 
group gives us two nonequivalent diperiodic groups. This, of course, 
requires the appropriate permutation of point coordinates and indices 
of forbidden reflections. 

Special positions of atoms with a fixed coordinate expressed as a frac- 
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tion of the unit-cell length in the z-direction, other than zero, are not 
allowed since fractions of a period are meaningless in this nonperiodic 
direction. 

The five nets (comparable to the 14 space lattices in three dimensions) 
for these diperiodic groups are the same as those for the 17 two-dimen- 
sional groups, namely, oblique (a ¥ b, y ¥ 90°), primitive and centered 
rectangular (a # b, y = 90°), square (a = b, y = 90°), and hexagonal 
(a = b,y = 120°), where y is the angle between the a and b axes. 

Alexander and Herrmann became interested in these groups because 
of their work with the smectic state in liquid crystals where only two- 
dimensional periodicity obtains. Cochran’s interest in them grew out of 
his use of ‘generalized crystal-structure projections” (Cochran, 1952, b) 
and Holser’s (1958, b) out of his investigation of the structure at the 
boundary between two parts of a twinned crystal (1958, a). 

The interest of the writer in making these groups available in con- 
venient form stems from cooperation with those members of Bell 
Laboratories who have been investigating surface structures by means 
of low-energy electron diffraction, in particular, L. H. Germer, J. J. 
Lander, A. U. MacRae and J. Morrison.* These structures are infinitely 
periodic in two dimensions but lack periodicity in the third dimension 
(normal to the surface). 

Which set of diperiodic groups is appropriate for surface structures? 
Certainly the structures are not strictly planar: the atoms of the surface 
structure in many cases do not all lie in the same plane. But would an 
atom above some plane (parallel to the surface) be symmetrically 
related to an atom on the other side of the plane? Strictly speaking the 
atoms could not be symmetrically equivalent since one is closer to the 
substrate than the other and is therefore in a different force field. From 
this point of view one would say that only the seventeen strictly two- 
dimensional space groups would be useful. However, it is frequently so, 
in triperiodic crystallography, that the symmetry of a crystal structure 
closely approximates a symmetry that is higher than its true symmetry 
and that the use of the higher-symmetry space group is of great help 
in determining the structure. From this point of view one would say 
that the 80 diperiodic groups in three dimensions are likely to be useful 
in the solution of diperiodic surface structures. Their application to this 
field was suggested to the writer by A. L. Patterson. 

There follow (7) a summary table, Table I; (iz) a diagram of net 
types, Fig. 3; (77) an explanation of terms and symbols used in the 


* For a survey of some of this work, see Low-energy Electron Diffraction, by 
A. U. MacRae, Science, 139, 1963, pp. 379-388. 


TABLE I—SuMMARY TABLE OF THE 80 DiIPERIODIC GROUPS 
IN THREE DIMENSIONS 


Net 


Oblique 


Rectangular 


Square 


Diper- 
iodic 
Group 
(DG) 
Num- 
ber 


IOP WN 


Full Hermann- 
Mauguin Symbols 


Pi 

PI 

P2i1 
Pmll 
Pbil 
P2/m11 
P2/6 11 


P112 

P112, 

C112 

Pilm 

Pila 

Clim 

P112/m 

C11 2/m 
P112/a 

Pil 2:/a 

P222 

P222, 

P22;2, 

C222 

P2mm 

Pmm2 

Pm2ia 

Pbm2, 

Pbb2 

P2ma 

Pam2 

Pab2, 

Pnb2 

Pnm2, 

P2ba 

C2mm 

Cmm2 

Cam2 
P2/m2/m2/m 
P2/a2/m2/a 
P2/n 2/b 2/a 
P2/m 21/m 2/a 
P2/a2;/m2/m 
P2/n 2/m 2;/a 
P2/a2/b 2;/a 
P2/m 21/b 2,/a 
P2/a 2,/b 2,/m 
P2/n 21/m 21/m 
C2/m 2/m2/m 
C2/a 2/m 2/m 


P4 

P4 
P4/m 
P4/n 
P422 
P42,2 
P4mm 


Triperiodic-Group Schoenflies Symbol, 
ITXRC Number and Orientation, if 
other than that given in ITXRC 


et 
COND Oe & WONDWNH 


joi, 


Ist setting 
Ist setting 
1st setting 
Ist setting 
Ist setting 


2nd setting 
2nd setting 
2nd setting 
2nd setting 
2nd setting tba 
2nd setting 
2nd setting 
2nd setting 
2nd setting 
2nd setting tba 
2nd setting cba 


bca 


bea 
tba 
acb 
aéb 
acb 
acb 
bca 
acb 


bca 
bca 
cab 
acb 


acb 
cab 


bea 


Symbol Pro- Weber 


db 
KP Nigeli 


1P1 
LPI 
1P2 
mP1 
aPl 
mP2 
aP2 


1P12 
1P12, 
1C12 
1Pilm 
1Plg 
1C1lm 
1P12/m 
1P12,/m 
1C12/m 
1P12/g 
1P12;/g 
1P222 
1P222, 
1P22,2; 
10222 
1P2mm 
mP12m 
mP12ig 
aP12ym 
aP12g 
1P2mg 
bP12m 

. bP12g 
nP12g 
nP12im 
1P2gg 
1C2mm 
mC12m 
aC12m 


mP2mm — 


aP2mg 
nP2gg 
mP2mg 
aP2mm 
nP2mg 
aP2gg 
mP2gg 
aP2gm - 
nP2mm 
mC2mm 
aC2mm 


1P4 
1P4 
mPA 
nP4 
1P422 
1P42,2 
1P4mm 


Num- 
ber* 
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TABLE I—ConTINUED 


























Diper- 
iodic at resnaces . 
G Full H 2 Triperiodic-Group Schoenflies Symbol Symbol Pro- Weber 
Net (HEP Mauguin Symbols TERRC Number and Orientation, it ose by Nu 
ber 
Square (cont.) 56 P4bm Ci, - 100 1P4gm 60 
57 P42m 4, - 111 1P42m 63 
58 P42,.m D3, - 118 1P42\m 64 
59 ae Bae LD le 65 
60 b2 Di,2i17 1P4g2 66 
61 P4/m2/m2/m Di, - 123 mP4mm 69 
62 P4/n 2/b 2/m D3, - 125 nP4gm 70 
63 P4/m2,/b2/m D8, - 127 mP4gm 71 
64 P4/n2:;/m2/m Dj, - 129 nP4mm 72 
Hexagonal 65 ae on - 143 i 49 
66 3 1, - 147 1P3 50 
67 P312 D3! - 149 1P312 54 
68 P321 D;? - 150 1P321 53 
69 esi Cae - 156 a 51 
70 31m 2” 157 1P31m 52 
71 P312/m Di, - 162 1P3lm 55 
72 P32/m1 D3, - 164 1P3m1 56 
oe: — s 
74 Ao m 73 
Pus GBB iit 
vere 
77 P6mm Ch, - 183 1P6mm 77 
Pe Gti ae 
9 m 3, - m m 75 
80 P6/m 2/m 2/m Di, - 191 mP6mm 80 








* Useful for cross-comparison of this list with those of Weber (1929), C. Hermann 
(1928) and Alexander and Herrmann (1929) since the equivalence among these three is 
given in the last reference. 


tables, Table II; (2v) samples of the systematic ITXRC “tables” for the 
80 diperiodic groups in three dimensions, adapted from the three-dimen- 
sional space groups by making the appropriate modifications; and (v) 
an annotated list of references. 

The full set of 80 diperiodic groups in three dimensions has been 
bound separately and is available from the Circulation Manager, Bell 
System Technical Journal, Bell Telephone Laboratories, Incorporated, 
463 West Street, New York 14, N. Y. It is anticipated that these groups 
will be included in a later volume of the International Tables. 


Ill. TABLES OF THE 80 DIPERIODIC GROUPS IN THREE DIMENSIONS 


In the sample tables, the usage and notation of the International Ta- 
bles for X-ray Crystallography for the three-dimensional space groups 
have been followed as closely as possible. Chosen directly from the 
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OBLIQUE RECTANGULAR, RECTANGULAR, 
PRIMITIVE CENTERED 


SQUARE HEXAGONAL 


Fig. 3 — The five nets. 


ITXRC Tables for the 230 space groups, these tables carry the same 
atom-position lettering. Letters of forbidden positions will therefore be 
missing. 

In the oblique and rectangular system, the ITXRC convention of list- 
ing the symmetry symbols in the order a, b, c has not been retained. 
Holser (1958,b) chose to permute these so that the first symbol referred 
to the c axis. The justification for this is that in the plane groups the c 
axis is unique and therefore should be put first as in, for example, the 
tetragonal system (e.g., 4mm). 

The possibility of confusion with the 230 three-dimensional groups 
will probably be avoided in all cases by the context. However, to aid 
in the distinction, the plane groups have been numbered, DG1, DG2, etc. 
The same letters could be used to distinguish DGPmm2, for example, 
from the three-dimensional C>2,! - Pmm2, but since, in all cases, the two 
groups do in fact comprise the same symmetry operations, such a 
distinction may be undesirable. 

The order of the DG list is that of the ITXRC which, in turn, is the 
Schoenflies order. 

After this paper was in galley form a communication was received from 
A. Niggli to whom a manuscript copy had been sent. Niggli favors plac- 
ing before the lattice symbol (P or C) that symbol referring to the glide 
plane or mirror plane which lies in the plane of diperiodicity and there- 
fore occurs only once. This occurs in 37 of the 80 groups. This would be 
another way of distinguishing these groups from the triperiodic groups. 
The symbol proposed by Niggli is also listed in Table I. 


80 DIPERIODIC GROUPS IN THREE DIMENSIONS 551 


TasLe I]—Sympous Usep 1n tTuE 80 DG TaBLes 





Symbol in the Symmetry Diagram 





Symmetry Elements eae 
Sears) Normal to the Paper Parallel to the Paper 

mirror m —— | vA ea 
glide plane* a | 

--- a 

n not allowed Z| 
2-fold rotation axis 2 6 ~— | 
3-, 4- and 6-fold rotation axes | 3, 4, 6 a @ 86 not allowed 
2-fold screw axis 21 not allowed = [ 
center of symmetry ] ° ° 
3-, 4- and 6-fold inversion axest | 3, 4, 6 A © G not allowed 
Center, on 2-fold axis 6 <o- ¢ 





_* This operation combines reflection with translation of 2 the length of the cell in the 
direction indicated by the letter. The diagonal glide, n, combines reflection with transla- 
tion of 3 of the length of the cell in both the a and 6 directions. 

{t Combined rotation through 360°/n (for %) and inversion. Not equivalent to the two 
operations performed separately. 


IV. EXPLANATION OF TERMS AND SYMBOLS USED ON THE 80 DG SHEETS 
(These are the same as those used in the ITXRC) 


1. Top of sheet, left to right: Net-type, full Hcrmann-Mauguin di- 
periodic group symbol, diperiodic-group (DG) number. The Hermann- 
Mauguin symbol begins with a letter which indicates whether the net is 
primitive or centered and is followed by symbols for symmetry elements 
that relate to the c, a and b axis, in turn. The c axis is normal to the 
paper in the diagrams, the a axis is directed toward the bottom of the 
page, and the b axis is directed toward the right. In DG 46 (P 2/n 
2,/m 2,/m), for example, the lattice is primitive, there is a two-fold axis 
parallel to c with a diagonal-glide plane normal to c, a two-fold screw 
axis parallel to a with a mirror plane normal to a, and a two-fold screw 
axis parallel to b with a mirror plane normal to b. In DG 16 (C' 11 2/m) 
we have a centered net with a two-fold axis parallel to b and a mirror 
plane normal to 6b. 

2. Diagrams: On the right, the distribution of the symmetry elements 
in the unit mesh. On the left, the distribution in the unit mesh of the 
points in the “general position” (x, y, 2 and points symmetrically 
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equivalent to it). Here, the value of x is taken, arbitrarily, to be a very 
small distance and y, a slightly larger distance, except in the oblique 
groups where the reverse choice has been made. The sign of ¢ is indi- 
cated beside the “point” (small circle). In both diagrams, the +2 
direction (a) is down the page, +y (b) toward the right. A comma 
within the circle indicates that that point is of opposite handedness to 
the points without commas, as when derived from these by mirror plane 
or inversion operation. Where two points are related by a mirror lying 
in the plane of the paper, half of the circular symbol is marked with a 
comma, half left blank. 

Below the diagrams are the lists of all possible points in this diperiodic 
group and equivalent point positions. 

First column: Number of positions that are symmetrically equivalent, 
given the first position in the series. 

Second column: Arbitrary identifying letter, conventionally the same 
as that first used by Wyckoff for this position. 

Third column: The symmetry of each point in the group (if each 
point lies on a two-fold axis, this will be “2’’; if each point lies in a mirror 
plane this will be ‘‘m’’; etc.). This will always be “1” for the “general 
position” which, by definition, is the position of a point not lying on 
any symmetry element. 

Fourth column: Coordinates of equivalent positions. Note that not 
every group has “special positions.” Special positions occur when a 
particular value of x, y, or z results in a reduction of the number of 
equivalent positions due to symmetry. 

Fifth column: Conditions on hk which must be satisfied, for x-ray 
reflection to occur when the point positions in column 4 are occupied. 


(References on page 559) 


Oblique P211 DG3 
+O) O+ 


C+ 


7! 


Ot +C) 


Number of positions, . ; a 
Wyckoff notation, Co-ordinates of equivalent positions 
and point symmetry 


Conditions limiting 
possible reflections 


_ General: 
2 e Le See Byes Rk: 
h0:+ No conditions 
Ok:) 


- Special: 
. No conditions 


aco am 
bo bo lO dO 
CO OnwrHNFK 
OilH NIH 
wR RR 


~~ 


Sa ee 


~~ 
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Rectangular C11 2/m DGI16 





Number of positions ; . cA diti imitin 
a cee notation, Co-ordinates of equivalent positions pet ie lta : 


(0,0,0; 2,2,0)+ 
General: 
8 j Lb 23522 292s. “82; 72: hk:h +k = 2n 


Special: as above, plus 


4 t m 2,0,2; #,0,2 ae 

4 2 0,y,0; 0,9,0 }No extra conditions 
4 e I 4,4,0; 4,2,0. hk: h = 2n; (k = 2n) 
2 b 2/m_—O,3,0. \ eh 

2 a  2/m 0,0,0. No extra conditions 
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Rectangular P 2/n21/m 21/m DG46 


+Q @+ +C2) @+ . | 2 





+Q) Ort +0) Or | | 


Number of positions, 
Wyckoff nota- 


Conditions limiting 
tion,and point 


Co-ordinates of equivalent positions possible reflections 


symmetry 
General: 
8 g 1 X,Y 25 £,Y,2; 3 — Lye a Y,e3 3 = 2,4 + Y 523 hk: h + k = 2n 
hO: (h = 2n) 
Zy,2; £,9,2; gt2,¢+y,2; 3+ 2,3 — yz. Ok: (k = 2n) 
Special: as above, plus 
4f m_ 20,2; £,0,2; 3 — 2, 4,2; 44+ 2,4,2 ie 
2 : : no extra conditions 
4 ¢ m 0,y,2; 09,25 3,3 — ¥, 2; 23 + Yy,2 
ee +40; 24.0; are 7,4,0 hkl: h = 2n; k = 2n 
1 105 ; 
: a git ete ‘no extra conditions 
ym y@) 292 .6@° 
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“Ss JIN YZ 


Number of positions, 


4 
4 
4 
4 
2 
2 
2 


Co-ordinates of equivalent positions 


Ks 


ry 
wy 
we 


2, OS 
G1 Os 
x Oo 


~ 
we 


So Bw 


we 

ble 
or 

Me 


2. 
~ 


Oxi- 
az 


Ee bp NNN 
Qin 


Q o> 


~e 
~ 


(om) Orne 
> 


fas) 
BIH 


a 
° 


YIN ZIN- 2 


Conditions limiting 
possible reflections 


General: 
No conditions 


Special: 

No conditions 
hk:h + k = 2n 
\No conditions 
hk:h + k = 2n 


\No conditions 


Hexagonal 


Or 


Or 


Number of positions, 
Wyckoff notation, 
and point symmetry 


3 d 1 
1 Cc 3 
1 b 3 
1 a 3 


Or 


Or 


Co-ordinates of equivalent positions 


Yet — Y, 2; Y— 2%, z. 
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DG65 


Conditions limiting 
possible reflections 


General: 
No conditions 


Special: 
No conditions 


Hexagonal P6 DG73 


#© at +O Siar 


4+ 
409 Or 
O 
+ 6 OF 70',.0% 


+ } 

Number of positions ae sob 
Wyckoff notation, ’ Co-ordinates of equivalent positions Seon peered 
and point symmetry possible reflections 

General: 
6 d 1 sx,y,2; J, — Y,2; Y — 22,2; No conditions 
£,Y,2; YY — ©, 2; L— Yye, z. 

3 c 2 7,0,25 042,25 253,2. : 

2 b 3 4,3,2; zi iz. Special: _ 

1 a 6 0,0,z No conditions 
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